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PREF AGE 


The present book aims at investigating the logical founda- 
tions of the theory of Markov random processes, 


The theory of Markov processes has developed rapidly in 
recent years, The properties of the trajectories of such 
processes and their infinitesimal operators have been 
studied, and intimate connexions have been discovered 
between the behaviour of the trajectories and the properties 
of the differential equations corresponding to the process, 
These connexions are useful for studying differential equa- 
tions as well as Markov processes, The material thus accum- 
lated has made necessary a critical survey of the fundamen- 
tals of the theory, In particular, the usual statement of 
the Markov principle of "absence of after-effects" has been 
found to be inadequate and various authors have proposed 
different forms for a strengthened principle whereby a pro- 
cess is "strictly Markov." It has become obvious that the 
most natural subject for study is presented by Markov pro-~ 
cesses cut off at a random instant, All these and other 
ideas were originally introduced by different authors in 
different forms, according to the specific purposes of their 
specialized works - in which stationary Markov processes are 
considered almost exclusively, 


A general theory is built up in the present book which also 
covers non-stationary processes, Stationary processes are 
regarded as an important special case, Non-stationary pro- 
cesses are well known to te reducible to the stationary type 
by an artificial method requiring the passage to a more com- 
plicated phase space*), However, the stationary processes 
thus obtained are in a certain sense degenerate, so that 
this type of reduction is by no means always suitable, Then 
again, a concept of the Markov process that is more general 
in essence is closer to first principles than the concept of 
stationary Markov process, There is a canonical time scale 
for stationary Markov processes, In the general process 
there is no such scale and all the definitions have to be 


*See article 3 of Chapter 4. 
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invariant with respect to any monotonic continuous trans- 
formation of time, 


The theory cannot be adequately developed by extending the 
concept of a Markov process as a random function of a special 


type. 


For we are usually concerned, when studying Markov pro- 
cesses, not with a single probability measure but with a 
whole collection of such measures, corresponding to all the 
possible initial instants and all the possible initial 
states; in other words, we are concerned, not with one ran- 
dom function, but with a whole collection of such functions, 
with definite inter-relationships., This is one of the rea- 
sons why the theory of Markov processes has to possess its 
familiar autonomy with respect to the general theory of pro- 
bability processes, The theory of Markov processes is built 
up in the present book without any reference whatever to the 
general theory of probability processes, 


This book cannot be used by the student to make his first 
acquaintance with the theory of Markov processes, Although 
we have not assumed formally any previous acquaintance with 
the theory of probability, in fact a reading of the book 
can only prove of value to someone already acquainted with 
an elementary exposition of the theory of Markov processes, 
such as is contained, for instance, in Feller's "Introduc- 
tion to probability theory and its applications," Vol, 1 
(Vvedenie v teoriyu veroyatnostei i ee prilozheniya), or 
Gnedenko's "Course of probability theory" (Kurs teorii 
veroyatnostei). 


The first introductory chapter contains a brief survey of 
the necessary concepts and theorems from measure theory. 
Any proofs that can be found in text-books are omitted here, 
The second chapter gives a general definition of Markov pro- 
cess and investigates the operations that make possible an 
inspection of the class of Markov processes corresponding to 
a given transition function, The more complicated operation 
of generating a subprocess is studied in Chapter 3. The con- 
nexion is revealed between the subprocesses of a Markov pro- 
cess and the multiplicative functionals of its trajectory. 
The most important classes of multiplicative functionals and 
subprocesses are investigated. Chapter 4 is concerned with 
the construction of Markov processes with given transition 
functions. The concept of strictly Markov process is 
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discussed in Chapter 5. Finally, Chapter 6 is devoted to a 
study of the conditions to be imposed on the transition func- 
tion so that among the Markov processes corresponding to 

this function, there should be at least one, all the traject- 
ories of which possess some type of continuity or bounded- 
ness, The supplement describes some of Choquet's results 
concerning the general theory of capacities, Measurability 
theorems for the instants of first departure are deduced from 
these results, A historical and bibliographical index will 
be found at the end of the book, 


The present work is closely allied to a monograph now in 
the press entitled "Infinitesimal operators of Markov pro- 
cesses" (Infinitezimalnye operatory markovskikh protsessov), 
which is devoted to the task of classifying Markov processes, 
The two works should be regarded as the two parts of a single 
monograph on the theory of Markov processes, 


The present material comes from a series of papers and 
special courses given by the author at Moscow and Pekin 
universities, The author is grateful to his audience for a 
number of observations which he made use of during the final 
preparation of the manuscript. 


I must express my indebtedness and sincere gratitude to 
Mr, A.A. Yushkevich for his careful reading of the manuscript 
and various comments that made it possible to eliminate a 
number of inaccuracies and obscurities, 


E.B, Dynkin 


CHAPTER 1 
INTRODUCTION 


1, Measurable Spaces and Measurable Sets 


-1.1. Leto# be a system of subsets of a set @ satisfying the 
following conditions: 


If AGoM, then A€ of *). 
If A;\oM(i = 1, 2, .6.), then YAEo%# ana 
1 


1.1.A,. 


atetedoe 
IVA:E of. 
1 


We say in this case that the system of subsets o#is a 3- 


algebra in the space @. Let C be any system of subsets of 
Qo. The intersection of all the o~algebras in the space 2 
that contain @ is also a c~algebra, We refer to it as the 


omalgebra generated by @, and write it as 5(@), 


If cis a c-algebra in the space wy and SE of, the aggre- 
gate of all sets A€of# that are contained in & forms a :- 
algebra in the space &. We shall denote this c-algebra by 


of {2}. 
We shall call the system of subsets C of the space @ a 
m=-system ifs 


1,1,B It follows from A, 4,€ @ that A,N AEC **). 


1° 
We shall call a system ¥ a \-system if it satisfies the 
following conditions: 


1,1.0,. QF. 


wi denotes the complement of A in Q, 1.e. QA 
**The intersection of sets A and B will generally be written 
as ANB, 
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1,1.0,, If Ay A€# and Ay 4,=—G *) then A,U AEF. 
1.1.0,, If 4, A€Fand A, PA,, then AN AER 
1.1.0). If A,,...,4y..-€& and A, tA**)then ACF. 


It may be noticed that, if a system of setsoMis simltan- 
eously a n-system and a \-system, it is a sc-algebra, For 
1,1.4, follows from 1.1.0, and 1,1.0;. Moreover, it follows 
from the relationship Ay 8=AUY(BN\AB)and properties 1.1.8, , 
Lele C, and 1,1, C, that, if A. BEo#, then Ae = AUe SAB) 
€ of, and vies Lt An AnicsceA n€ of then U A;€ co. Now 
let A, n€ohk (n=1, 2,...). Then AU ALG of ? and inasmuch as 

oO. 


At , we have U Eo# by 1.1. Oy It follows from the 


relationship 
N4.=U4n 
1 1 


that Q An,€oM - Condition 1.1.4, is therefore satisfied. 
1 
Lemma 1,1, If the \-system ¥ contains the 7-system C 
then # contains o(@€), 


Proof, The intersection ¥’ of all the \-systems contain- 
ing the xs-system € is obviously a \-system., We shall show 
that this intersection is at the same time a t-system, The 
assertion of the lemma will follow from this, 


The class #1 of all the sets A such that An BE’ for all 
BE Cis readily seen to be a \-system, Since ¥,>@€, we 
have ¥,>¥/ This means that, if AC#’. BE@ , then AN 
BEF’, 


We now put BE F,ifBnAAC¥ for allAC¥’. It may readily 
be seen that #> is a A-system, By what has been proved, 


*The symbol 8 denotes the empty set. oo 
**A,+A means that 4,G4.6¢...G4,¢... and A=[J4n,. Simi- 


n=l 
larly, 4n)A means that A;D4,>...24n=>.-.. and = 
A=[)] An. 
1 
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Fo2C, 80 that ¥,.2F’, This means that, if A BEF’, 
then AN BEF. Consequently ¥’ is a t-system, 


1.2. The pair (2, ), consisting of a set 2 and agi- 
algebra of of subsets of 2 is called a measurable space, 


An important example of a measurable space is provided by 
the space (Jj, @i) , where =|[s, t] is a segment of the real 
line and #t is the c-algebra of subsets of this segment 
which is generated by all the intervals contained in fF. 
The values of s and ¢ can be infinite as well as finite, and 


we put 1,3 =(—0o, +00), = (— 00, t), Meo=ls, +00) » 


Let (&;, €,) and (Q), uf.) be two measurable spaces, The 
mapping of the space Q, into @, is said to be measurable if 
the complete pre-image of any set from uf, belongs to #1, 
This definition is likewise applicable when the mapping is 
defined only on a subset @,cQ., 


It may be noted that, if «o(@)=.4#,, the only requirements 
that need to be satisfied by the mapping for the reflection 
ato be measurable are that the complete pre-image of any 
set from @ belongs to vt; and that: = Ucn , Where C,E C 


= 
(this is proved simply by observing that the sets whose con- 
plete pre-image belongs to 4: from a c-algebra in &,), 


Obviously, if a is the measurable mapping of (Q,, £,) into 
(2,, 2) and 8 is the measurable mapping of (2, £,) into 
(2, 43) , then fa is the measurable mapping of (Q,, 4,) into 
(Q5, fs) , provided only that the domain of definition of 8 
contains a(Q&,). 


The most important particular case of mappings is provided 
by numerical functions, i.e, mappings into the real line ia. 


Let ux£ be a s-algebra of subsets of @&,. We shall say that 
the numerica: functions §(w)(w€2) is measurable with respect 
to or is #-measurable, if the mapping defined by it of 
(2, vt) into (33, Pis)e is measurable, i.e, if for any 

TE Bro ? 


{ws E(w) ET) € of. 


Since the intervals (f,-+00) generate the c-algebra @72 , 


the requirement for the function E(w) to be of-measurable is 
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simply that, for any ¢, 
{w :&(w) > t} Ef. 


The concept of 4~measurable function extends automatically 
to the case when the system of sets uf is a o-algebra in some 
subspace © of the space @ instead of in the whole of ©, 

It may easily be seen that in this case the domain of defini- 
tion of any uf -measurable function & coincides with 9, 


Let £ be any system of real functions in @, satisfying the 
condition 


1.2.4. If t€ Lana 


wa[tO rimto>o 
ce | 0 with E@) <0, 


then n andt - 7 belong to Y, 
A system of numerical functions o& is called an .?-system if 
the following conditions are fulfilled: 


1,2.B,.1€ &. 


1.2.B,, A linear combination of any two functions of HC 


also belongs to #, 


1.2.8, If §&€ 5, 0<£&,(w) t&() *) and&(w) is bounded or 
belongs to 2, then t¢H, 


Lemma 1,2, If an .#-system 6 contains the character- 
istic**) functions of all the sets belonging to some 
r-system C, then d contains all the functions of ¥ which 
are Measurable with respect to s(@). 


Proof, The class of all sets whose characteristic functions 
belong to forms the A-system ¥, Since ¥ DC, we have 
oF 2°(C) by lemma 1.1, 


*We write anta ifa,+aand a<ac<...<¢a,<... Similarly, 
an¥@2 means that a,—>a and 4>4.>..u>a,>.... 
**The characteristic function of a set A is the function 


wafh ifeca 
1 0, ifaea. 
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Let — be a non-negative function belonging to Y and 
measurable with respect to o(@). We put 


k 
Tin ={  <E (6) < <+*}, 


1 
22n 
Rk 


= 2% XP pn 


k=0 


Obviously ,O<&, f& and, in accordance with 1,2.B,, cCH. 

In view to 1.2.4 an arbitrary >(@)-measurable function 
n€ can be represented as the difference between two o(@) 
-measurable non-negative functions of ., The latter belong 
to #, as already pointed out, Consequently, we also have 


NE GB. 


1.8. Let 4; be a s-algebra of subsets of sets Q, (=1, 2, 
.... 1). We shall write Q.x...x@, for the set of vectors 
(Wj, ..-, On) 9 Where o,€&;, , and, X... Xu, for the o~algebra 
enerated by subsets of the form A,X ... XA, , where A4;€ 4; 
(it may be noticed that the sets A,X ... XA, form a ¢-system), 


In the case when Q,=...=—=2,=—@, we shall write 2” instead 
of Q.x... XQ, , and similarly 4" instead of fi:X ... Kuen, 
when =... = #, = dc. 


Now let an infinite sequence of spaces @; be given and a 
c-algebra of subsets vt; in each 2 space, We shall write 
2x... X&2,xX... ror the space of sequences (4, @, ..., 
Wa ss.) 4 Where w,€Q; , and uli X... X4nxX.-.. for the o- 
algebra in this space generated by the subsets 


Ay X Ap X ... MAX Seg Kh Sins x see (1,1) 
(#1, 2,...; Ap€ vf). 


When the factors are the same we shall write 2@ and 
for brevity. It may be noticed that the class of subsets of 
type (1.1) is a x-systen, 


Lemma 1,3. Let a, be the measurable mapping of (2, -4) into 
(Q;, v3) (i can run either through the values 1, 2,....2”, or 
through all natural numbers). Then the mapping o of the 
space (2, #) into (2,x2,x%..., Xu ...), defined by the 
expression 


& (w) == {a1 (), a (w), ...}, 


is measurable, 


6 THEORY OF MARKOV PROCESSES 


Proof, Suppose for clarity that i runs through all natural 
numbers, The class of all the sets of %#,X 2X .-- , whose 
complete pre-images with mapping « belong to 4, clearly 
forms a c-algebra. This sc-algebra contains all the sets of 
type (1.1). It therefore contains 4,%%X..... 


Lemma 1,4, Let .£; be a c-algebra of subsets of the space 
Q; (i= 1, 2) and let f(«,, ws) (w,€ Qy, w EQ.) be an vf, X uf, -measura~ 
ble function, Then for any fixed o,€&, f(m,, 2) is an 4, = 
measurable function of w,, 


Proof, Let & denote the class of all functions in the space 
2,2, . The system # of all functions f(;, »,) for which 
our lemma holds is evidently an .Y-system, This system con- 
tains the characteristic function of any set A,X A2, By 
lemma 1,2, it contains all functions measurable with respect 
to the c-algebra 4, xX cho. 


Lemma 1,5. For each ¢ of some set T let there be associa- 
ted a function x,(w) (o€ 2) with values in the measurable 
space (£, #),. The necessary and sufficient condition for a 
function £(w) (w€ Q) to be measurable with respect to the s- 
algebra Wr generated by the sets {w: x,(w)ET} ((€7, TE gw) is 
that " 


E(w) =f [xz,(w), x2,(W), ...5 Xt, (), ..-], (1.2) 


where (ty ty ....éy.-. ET and f(x, x, ..., X...) isa - 
measurable function in the space E”, 


Proof, The mapping of 2 into the real line /;Qj=(—o, +09) 
given by expression (1.2) can be written as the product fa, 
where 


a (w) = {%2,(), Xt, (w), eee a (w), eee |. 


For an} ¢€T x,(w) defines a measurable mapping of (2, Wr) in- 
to (£, #). By lemma 1.3, « is the measurable mapping of 
(2, oz) in es 7 . By hypothesis, f defines the measurable 
mapping of (E*. g”) into (/[%. #73) - Consequently §=/fa 
is the measurable mapping of (2, Wr) into (72. #72). Hence 
all the functions that can be written in the form (1.2) are 
aN 7 -measurable, 


Now let % denote the class of all functions £(w) (w€Q), 
let # be the set of functions § that can be written in the 
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form (1.2), and let @ be the system of all w-sets of the 


form 
{x2,ET, re x: EP} 
(n= 1, 2, 0005 ty wees te ETS Dye. .) Te€ F). (1.3) 


The characteristic function of the set (1.3) is equal to 


Xr, [*,, (w)] oe om {*,,(@)]- 


Thus it can be written in the form (1,2) and belongs to #, 
Obviously, 6 is an Y-system, and @ a x-system, and by © 
lemma 1,2, contains all the functions §(w) measurable with 
respect £0 s(C)=NT, 


2, Measures and Integrals 


1.4. Let (2, of) be a measurable space, The non-negative 
numerical function 9(A) (A€of) is called a measure*)if, for 
any finite or denumerable collection of mutually disjoint 
sets A;, Az --- of of, ?(UA,)=2e(A,) . The measure satisfying 
the condition o(2)=1 is known as the probability measure. 


Let » be a measure given on the s-algebra of, Let f be an 
od-mneasurable numerical function, given on some subset 2, 
of the space @, and let A be contained in @,, We shall 
say that the function f is g¢-summable in the ‘set A if the 
finite limits exist, 


— 


+00 
F R 
im Dn t{riees. Z<W<e 
0 


*All the statements and definitions contained in section 1.4 
can be translated without difficulty to the case when the 
function ¢(A) is allowed the value +o as well as finite 
values, (A particular example of such.a function is the 
ordinary Lebesgue measure on an infinite numerical straight 
line.) We shall only encounter such functions in a few 
examples, however, so that, unless there is some special pro- 
viso to the contrary, the measure ? will always be uoerneeeed 
to be a function that takes only finite values, 
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and 


lim pitts {wrweA, Ze fw) <4 \*), 


nay 


The sum of these limits is called the (Lebesgue) integral 
of the function f on the set A with respect to the measure ¢ 
and is written as 


f fF) ¢@e). 
A 


If one limit is infinite but the other finite, the inte- 
gral is ascribed the valuectoo (+ooif the first limit is 
infinite, and—ooif the second limit is infinite). 


We shall assume that the reader is familiar with the basic 
properties of Lebesgue integrals, The properties that we 
shall use most frequently are as follows: 


1.4.4, If0</s,(w)tf(w)for all w€A, then 


lim J fu (w) ¢ (do) = J F(w) 9 (de). 
3 J { (1.4) 


1.4.B, If for all w€A f,(w)>f(o), |f,(o)|<g(w)and g is ¢- 
summable in A, then 


lim | fy()@ (dw) = | f(w) ¢ (do). 
A J (1.5) 


1.4.C, (Fubini's theorem.) Let of; be a c-algebra in Q, 
and 9; a measure in of; (i=1,2). Let /(w;, w,) be an of, x of,- 
-measurable function in @, « 2, such that 


i flf@ W.) | Po (do 9, (dw) << 00. 


2, 
J [fos 2) 92 (dos) ©, (dw,) = 
=f Jie 2) P, (da) | G2 (dw.)**) , 


*When a ?-summable function is referred to without mention- 
ing on which set A, it will be a question of a function which 
is ¢-summable throughout its domain of definition, 

**If f is non-negative, the proviso regarding the absolute 
convergence of the integral on the left-hand side of (1.6) is 
unnecessary, 


oO 
=2 


(1.6) 
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The following property is a particular case of 1,4,A: 
1.4.4,. If A,fA, then 9(A,)t9(A). 
1.5. We shall prove two useful lemmas, 


Lemma 1.6, Let 2 be the measurable mapping of (Q,, v4;) 
into .(Q,, xf.) and let o be a measure on .4,, Then the expres- 


sion 
vT)=g{aEr} MWeA#) (1.7) 


defines a measure in f2, We have for any .4,-measurable 
function f: 


ff 4 de) = f fla(o] ¢ (do) (1.8) 
Q, 2, : 


(more precisely, if one of these integrals exists the other 
exists and the two are equal). 


Proof, The first part of the lemma is obvious, 


Let Y denote the class of all ’-summable functions, The 
class of all functions for which equation (1.8) is satisfied 
is clearly an %-system, This system contains the character- 
istic function of any set of 42. By lemma 1,2, it contains 
all the .4,-measurable functions of 2%, Consequently, if the 
integral on the left-hand side of (1,8) exists for f, f 
satisfies (1.8). It can be shown similarly that (1,8) is 
satisfied for any function f for which the integral on the 
right-hand side of (1.8) exists. 


Lemma 1.7. Let U, V, Z be three spaces and dy, uty, 42 
o-algebras in these spaces; letF(u, z)(a€U,zE€Z) be a real 
function measurable with respect to éuXéz 3; and let 
P,(v€V) be measures in the so-algebra véz, the numerical 
function P,(T) being véy-measurable for any YTé€4z. If the 
integral ” . 


G (ua, v)= i F (a, 2)P, (dz) 
Zz (1.9) 


converges for all w€U, v€Y it defines an uy X tr- 
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measurable function*), 


Proof, Let % denote the system of all ty X ulz -measurable 
functions F(z, z) for which f F(u, z)P,(dz) converges for all 


Z 
u€U, vEV . The system 4 of all the functions F(u, z) for 
which G(z, v) is ufo X #v-measurable is evidently an Y- 
system, It contains the characteristic function of any set 
Ay X Az(Ar€ Ay, Az€ vez). Since these sets form a x-systen, #7 
contains all %y X uéz-measurable functions of Y by lemma 
Lees 


3. Conditional Probabilities and 
Mathematical Expectations 


1.6. The triple (2, o%, P), where 2 is a set, of is ac- 
algebra of its subsets, and P is a probability measure in -o, 
is called a probability space, The points of @ are called 


element events, the elements of o# are events, and the 
values of P(A) are their probabilities, Every o#-measurable 
function £(w) (w€ 2) is called a random variable. The 


integral f E(w) P (dw) (if it has a meaning) is called the 


expectation of § and is denoted by Mi. We shall also des- 
cribe as random variables o#-measurable functions § which are 
only defined in some subset 2 of @ instead of throughout 
the space @ (it follows from the o4-measurability of & that 
Q.€ o#), The expectation of sucha {is given by the expression. 

ME = | E(w) P (dw). 

2 
Let Ge2 and let p(w) and p(w) be two functions whose 

domains of definition contain ©. We shall say that p=} 
almost certainly in © (in the sense of the measure P) and 
write - 

g=y @.c. g, P), 


if P(S.9+#)=0. Similar meanings attach to the expressions 


P< (a.c.&, P), n>? (a.c .Q, P) 
and so on, 


*In the case of a non-negative function F the requirement of 
convergence of.the integral is unnecessary, 
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Let % be a c-algebra in G2e@, where £204, Let the 
function §(w) be P-summable on the set 2, Every .f-measur- 
able function satisfying the relationship 


E(w) P(dw) = | M (E44) P (do). 
j J (1.10) 


for any A€.£ is known as a conditional expectation of .£ with 
respect to # (written as Mé€].4)). 


Such a function may be shown always to exist, It may easily 
be seen to be defined only up to an arbitrary set of the o- 
algebra having P-measure zero, Since it is impossible 
to conclude from the equations M (|-4)=9, M(&|.4)= that 
¢(w)=$() for all w, the only conclusion that may be dram 
being that ¢= (a.c. @, P ), we shall avoid paradoxes by 
writing M(|4)=9 (a.c. §,p) instead of M(E].f)=o. 


It should be noticed that the function M(&]|-4) is unaffec- 
ted by the values of the function § outside §&, 


If B€o#, the characteristic function Xz() is an - 
measurable function, The function M(y,|v€) is called the 
eonditional probability of the event B with respect to the 
c-algebra st and is denoted by P(8|£). This function can 
also be characterized as the vf-measurable function satisfy- 
ing for any A€ 63 


P (AB) = | P (BJ 4) P (du). 
J | (lid) 


(It is defined only as far as an w-set of # having P- 
measure zero,) 


We shall mention a few examples*), 


1.6.1, Let % be an algebra consisting of two elements: 
the empty set 9 and the whole of the space ©, Then con- 
stants are the only .#, -measurable functions and we have 
from relationship (1.10) with 4=@. 


M (&| 5) == ME 
*The g-algebra £ does not contain non-empty sets of measure 


zero in examples 1.6.1 and 1.6.2, so that the function P(&jt) 
is uniquely defined for any §&. 
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1.6.2, LetA€o# and let O<P(A)<1. The four sets 


{0, A, A, @} form a c-algebra £. We have 


pia | Eo) P de) for w€ A, 


A 
Ml) = 
1 sy 
way f HOP am for EA, 
( P(BA) 
P (A) for w€ A, 
P(B] 4) = P (BA) ba aee 


These expressions establish the connexion between the 
general concepts of conditional mathematical expectation and 
conditional probability with respect to a given o-algebra 
and the popular elementary concepts of mathematical expecta- 
tion and probability that some event A might occur, 


We shall deduce the fundamental properties of conditional 
mathematical expectations and conditional probabilities. In 
the statement of these properties, .£ denotes a c-algebra in 
GeQ , it being assumed that £coo#. 


1.6.4. If § is -measurable and P-summable in 2, then 
MC[ ef) = (a.c. &, P). 
1 6.A,. For any constant c, 
M(cluf)=c (a.c.&, P). 
1,6.B. If & and 7 are P-summable in © and 
E> (a.c.8, P), (1,12) 
then 
M(|4)>M(n[-4)  (a.c.8, P). (1,13) 
1,6.B,. For every A€ of 
0< P(A <1 (a,c. P). (1.14) 
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If P(Q@\ 48)=—0, then 
P(A]. 4)=1 (a.c.& Py, (1.15) 


1.6.c. If as and § are p-summable in ©, c, and c. being 
arbitrary constants, then 


M (C18: + coke | xf) = CM E,[-C)+eoMG|4) @.c.2, P). 
1.6.D, If § is P-summable in © and 
0<e, Té (a,c, Q, P), 

then 

ME,fe) tMEfu£)  (a.c.9, Py. 
If A, tA, then 

P(A,J4)tP(Afé)  (a.c.9, P). 
1.6.E, If 7 is P-summable in © and 


1.6.D,. 


[Sep Heekees ea ees ons (a.c,9, P), 
then 
ME, | @—>M(El4) (a.c., P). 


1.6.F. If & and 7 are p-summable in © and § is £- 
Measurable, then 


M(&q|-t)=§M(n]£) (ac. ©, P). 


1.6.G, Let 259,56,, let of, be a c-algebra in ©, and 
wt, a spalgebra in G , where 4,S90%, -f,E 0% and 4,9, EAs 
for all 4,€c#,*) . “If the functions fy and 7 are ‘P -summable 
in 9, and the function & is equal to zero in o.\, and 
induces anvf,-measurable function in Q@,, we have 


MEn| 4) =MEM (ft) feil  (@-c-2, P) **). (1,16) 
If A€ut, BEo#, then 


P(AB| 4) =M{x,P(Bft.)f4,}  (a.¢-8,, P). (1,17) 


*That is, 4, (%,)Ct,3 in the case when @ .5,, this require- 
ment reduces to the simpler condition 4,C.4.. 

**The first factor in the product §&M(nfJ4.) vanishes whilst 
the second is not defined in %,\9& . The product is 
reckoned to be defined and equal to zero in 4%,\ 9). 
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1.6.H, If the functions & and 7 are P-summable in © and 
the function is vf-measurable and equal to zero (or not 
defined) outside Q, then 


MEy = M [EM (n] v2)I- 


1.6.1. Let be a c-algebra in the space ©, If yis P- 
summable in @, then 


If A€o#, then 
P(A) =MI[P(A]-4)]. (1,19) 


We now turn to the proof of properties 1.6.A-1.6.1, 


Property 1.6.4 is an obvious consequence of the definition 
of conditional mathematical expectation, On putting E=c, 
in 1.6.4, we get 1.6.A,. 


Proof of 1.6,B,-1.6.B,. We have by (1.10) and (1.12), for 
any A€ut: 


M (Eu) P(dw) = [ &P (dw) > [ 4P (dw) = 
/ : >J (1.20) 


= f M(nJot)P (du). 
We put . 
A,={0:M Gof) <M(nfot)—>}. 
Obviously A,€vué and 


Jf MeLoP ao) < f mofo P@s)—1 P(A). (3,01) 
Ay Ay 


It follows from a comparison of (1.20) and (1.21) that 
P(A,) =0-. But 


{ME]t) << M(n]e4)} CUAy, 
whence (1.13) follows, 


To deduce (1,14), we only need to notice that 0<y,<1, 
and to make use of properties 1.6.B and 1.6.4, already proved. 


INTRODUCTION 15 


Finally, if P(Q\AS)=0, then x,>I1(a.c. g, # and, by 
(1,13) and 1.6.4, P(A] 4)=M(x,]t)>1 (a.c, §,P), Taken 
in conjunction with (1.14), this gives expression (1.15). 


Proof of 1,6,C, In view of (1,10), we have for any A€c#: 
fep (do) = { MG JOP Ge); 
A A 


ff EP (do) = { M&|€P (do), 
A A 


so that 
J abst cfs) P (dw) = f [cM G,]4)+-oM | O1P (da), 
A 


A 
which is equivalent to proposition 1,6.C. 


Proof of 1,6,D-1.6.D,. In accordance with 1.6.B, it 


follows from the relationship 
0<h< «1. Ci... =a.e. &, P, (1.22) 
that 
VIM JAS ... CMEJO<... (a,c. Q P). (1.23) 
By (1.10), for any AC 4, 
J MG.LeOP Go) = f EP (dw). (1.24) 
By 1.4.4, it aa from (1.22), ae) and (1.24) that 


f lim M(,Jv€) P (dw) = f lim &,,P (dw) 
A A 


and therefore 
MEn|£)=M(Efet)  (a.c. &, P). 


We can prove 1.6.D, simply by applying 1.6.D to the 
quantities 4, 


Proof of 1,6,E, We put 
a _ ° 
on = ou cask = jnt kate. 


Obviously, 
O<n—&[n—t§ & O<n+tG inte (a.c.&, P). 
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Therefore, by 1.6.D, 


M(n—Cife4)[M(n—tft) & M(n+orf 4) [Mm+eya) 


a.c,. G, P). 
Thus ( 


M(cr fe) | MEl@ & M(Crpot)TMEft) (a.c.8, Pp). 
It remains to remark that, by 1.6.B, 
M (C5 | 4) << ME, [) <M (CRI L2). 


Proof of 1,6.F, We fix any function 7. P-summable in @ 
and put §€.9, if & is P-summable in @. By 1.6.C and 
1.6,D, the class $ of all §for which 1,6.F is satisfied 
forms an f-system in ©, It follows readily from (1.10) 
that 6? contains the characteristic functions of all the 
sets of £4, By lemma 1.2, contains all the uf-measurable 
functions of 2%, 


Proof of 1,6,G, We have by (1.10) and 1.6.F for every 
AE ul,: 


f MEM Cn] of.) [o4:} P do) = J EM (| 42) P (dw) = 
A 


~ femenbtyP a) — J MGnf 4) P da) = 
AS, AQ, (1.25) 


= f EnP (dw) = J EnP (dw). 


Ag, 


Since the function M {&M (n]v4.)]vé,} is -£,-measurable, 
aaa follows from (1.25). On potting E=y,. n==y,» we get 
1.17 


We prove property 1.6.H simply by applying 1.6.G to the 
case O.=2, 4,=o%, &.=—8, #,—cf.. We get (1, 18) on 
ae ). O—O, t—1 in 1.6.H, and (1.19) by putting 1=y,- 

1,18 


4, Topological Measurable Spaces 


1.7. A topological space is a pair (£, @), where EF isa 
set and @ a system of subsets of EF such that: 
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1.7.A, . The empty set and the whole of space E belong to@, 


1.7.45. The intersection of a finite number, and the sum 


of any number, of subsets of C again belongs to C, 


The elements of the system C are called open sets of the 
space E£, Every open set containing a point x is called a 


neighbourhood of x. 


We describe AGE as closed if A€C, The intersection 
of all the closed sets containing a set Bis called the 
closure of 8, The closure of Bis the least closed set con- 
taining B, : 


Let x, be a sequence of points of the space (£. @), If 
every neighbourhood of the point * contains all the elements 
of the sequence *n apart from a finite number, x, is said to 
be convergent to x and we write *%,—-* , 


The topological space (£. @) is said to be compact if a 
convergent subsequence can be chosen from every sequence, It 
may easily be shown that a finite covering can be chosen from 
every denumerable covering of a compact space F by open sets*), 
If this condition is satisfied for any (not merely denumerable) 
covering of E by open sets, (E, @)is said to be a bicompact 
topological space, 


We refer to (E. @) as a Hausdorff topological space if, for 
any two distinct points x, y€E disjoint sets U, VE @,can be 
found such that +x€U, y€V. A bicompact Hausdorff space is 
called a bicompact. 


A subset A of a topological space (E, @) is said to be 
everywhere dense in EF if, for every non-empty CEC , the 
intersection ANC is non-empty. If a denumerable everywhere 
dense set exists in £, (FE, @) is described as a separable 
space, 


Let E’ be an arbitrary subset of E and let @’ be the class 
of all subsets EF’ expressible as intersections E’MC , where 
CE€@. Evidently, @’ satisfies conditions 1,.7.A,-1.7 Aas so 


that (E’, @’) is a topological space, Every subset of a topolo- 
gical space can thus be regarded as in turn a topological 


*)A system of sets @ is said to form a covering of —& if each 


point x¢€£& belongs to at least one set of @, 
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space, In particular, it is meaningful to speak of the com- 
pactness or bicompactness of a subset E£’, 


If E can be expressed as the sum of a denumerable number of 
compact (bicompact) subsets, the space (E, @)is said to be 


¢-compact (s-bicompact). If there is an open set U anda 


compact (bicompact) set K for every point such that x€UCK, 
E is said to be locally compact (locally bicompact). 


Let (Ej, @,) and (E, C2) be two topological spaces, Let @,x 
C2 be the class of subsets of the set £,XE, expressible as 
sums of "rectangles" C,XC.(C,\E C1, C.E @2) - The topolo- 
gical space (Fi X EF, @iX C2) is said to be the topological 
product of spaces (F,, @,) and (F,, @,). 


It should be noticed that (%p» va)>(*. y) in(~f: XE, C1 @,) 
when and only when x,—-¥%x in(E,, @,;)and jJa>y in (E,, @;). 


A metric space is defined as a pair (£, p), where F isa 
set and p a non-negative function in EXE, satisfying the 
following conditions: 


1.7.8,. p(x, yo=pely, x) 
Led eae Y+e Ur» 2)>p(*, 2) 


1.7.3 p(x, y)==0 when and only when x=y, 


3° 
The set of all points » satisfying the condition p(% y)<e 
is called an e-neighbourhood of the points x and is written 
U,(x). A set U is said to be open if, for every point x€U, 
there is an e>0 such that U,(x)G2U., The class @ of all 
open sets U satisfies conditions 1.7.4, -1.7.4,. Every metric 


space can therefore be regarded as a topological space and 
all the concepts introduced for topological spaces can be 
applied to it. Topological spaces which can be got with the 
aid of the above construction from some metric space are 
described as metrizable, 


A sequence of points x, of a metric space (EF, p) converges 
to a point x when and only when o(x,, x)>0. 


A metric space is said to be complete if every sequence x, 


satisfying the condition lim p(x,,, x,)==0is convergent. 
m, "> co 


The simplest example of a metric space is the set of all 
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real numbers R with distance p(x, y)=|y—-x|. This space is 
complete, separable, o-bicompact and locally bicompact, 


1.8. Let (E,, @,) and (F., C2) be two topological spaces, The 
mapping « of £, into E, is said to be continuous if the conm- 
plete pre-image of any set C€ @, belongs to @,. If a is the 
continuous mapping of (F;, @,;) into(F, @.)and B is the contin- 
uous mapping of (E;, C2) into (Ey @s3), fa is the continuous mapp- 
ing of (E,, @1) into (E;, @;). 


If « is the continuous mapping of (EF; @1) in (Ez Ge), it 
follows from 4*,—~.x that a(x,)—a(x). The question arises 
as to whether the fact that a(x,)-a(x) as x,-x implies the 
continuity of the mapping a. Although this is not in general 
true, it is true in the case of metrizable spaces. 


It follows from 1,7.B, that, if *,—>+*x, y,>y, then Pp (Xp Vn)> 
p(x, y). The function” p(x, y) is therefore continuous in 
the topological space (EXE. @X@) (@€ being the system of 
all open sets of (E, p))*). 


Let [ be a subset of the metric space (E.p), We write 
p(x, T) for the lower bound of p(x, y) over all yer. It 
follows from 1.7.B,-1.7.B, that 


le, T)—e(y. TM) |<p(x, y), 


so that p(x, Tl) is a continuous function of x. 


1.9. A topological measurable space is a triple (FE. C., #), 
where the pair (EF, @) is a topological space and the pair (£, #) 


is a measurable space, 


The simplest and most important particular case of a topolo- 
gical measurable space is provided by spaces of the form(E, @, 
o(@)). We shall sometimes write these as (F. C) for brevity. 
The elements of the oc-algebra o(@) are known as Borel sets of 
the topological space (E£, @). 


Let (£;, @1, #,) and (FE, Ce, #2) be two topological measura- 
ble spaces and a the mapping of F, into &,. It is meaning- 
ful to speak of a as being measurable or continuous, We shall 
show that the continuity of « implies its measurability if 
#,=2(C1) and #,=0(@,). For let # denote the class of all 
subsets of EF, whose complete pre-images belong to «(@,;). 


*)The space (£ XE, Cx@) is obviously metrizable, 
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Obviously, # is a \-system and contains the x-system @2, 
By lemma 1.1, F¥D<s(@,). 


There is no point in confining ourselves to topological 
measurable spaces for which @=«(@), since this considerably 
restricts the possible applications of the theory. Various 
assumptions will be made regarding the connexion between # 
and C , depending on the circumstances, The most important 
of these assumptions are stated below, 


1.9.4. # is a c-algebra generated by some sub-system of 
the system @, or what amounts to the same thing, #=c(#N@C), 


1.9.B. For every UEC #NE there is a #-measurable con- 
tinuous function f(x) such that f(x)#0 when and only when 
xE€U, 


Lemma 1,8, Let the topological measurable space (£. CC. #) 
satisfy conditions 1,9,A-1.9.B, If the -system § contains 
all the bounded continuous measurable functions in F, it 
also contains all the measurable functions belonging to 2, 


Proof, The system of sets #N @ is a x-system, where, by 
1.9.4, “«(#N@C)=#. In accordance with lemma 1.2, we only 
need to show that the characteristic function of any set 
UEPNCe hbelongs to &&. We take the function f(x) defined 
by condition 1,9.B and put 


i if |u|> 


? 


snaeal Fn (*) = Qn Lf (I. 


| lz), if jul< 


1 
n 
1 
n 


, 


The functions /, are continuous, bounded and measurable; 
they thus belong to #@. Since 0</f, ty, , we have %~€H., 


Lemma 1,9, Let %...., @,... be measurable mappings of a 
measurable space (2, o#) into a topological measurable space 
(E.C.#), satisfying conditions 1.9.A-1.9.B. If a,(w)->a(w) 
for any wE€Q, the mapping « is also measurable, 


Proof, Let VE CN# and let f be the function defined in 
condition 1.9.B. We have 


Co CO © 


MEU=Y U A ilfteoul> Zz}. 
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The function f[a,(w)] is measurable, so that {|/[z,(w)]|> 
>z Eo. Consequently (a(w) CULE of . It may easily be de- 
duéed from this, by 1.9,.A, that {a2(w)€T}Co# for every TE#. 


A term may be introduced that we shall occasionally find 
useful below, 


We shall say that the points ti define a canonical sequence 
of subdivisions {a?} of the interval A if: 


1) for any r,A=[JApA? and A? being disjoint for i+; ; 
k 


2) the points “ is the right-hand end of the interval 
At 3 all the points #2 belong to A; 


° % 
3) lim sup | At] = 0 where |Az|=|#2—f#_,| is the length of 
the interval Aj, 


Lemma 1,10. Let A be a numerical interval, (2. o#) a measur- 
able space, and (FE, @, #) a topological measurable space 
satisfying conditions 1,9.A-1.9.B. Let the mapping F of the 
product AX@ into E satisfy the conditions: 


a) for any 14¢€A, the mapping Ff is a the measurable mapp- 
ing of (2, of) into (E, #) 3 


b) for any w€2, the mapping F is the mapping, contin- 
uous from the right, of a into (F, @)*). 
Then F is the measurable mapping of (A x &, #, x of) into 
(E, #) e 
Proof, Let the points #; define a canonical sequence of 


subdivisions {A?} of the interval A. In view of a), the mapp- 
ings F, of the space AX@ into E are given by 


F(t, ©) =F (é%, o), if té Ay, 


and are measurable, In view of b), 
F,(t, o) > Fé, o) 


ag nm-—->oo and F is measurable by lemma 1.9. 


A metric measurable space is the triple (EF, p, #) , where 


*)That is, for any ©€9 and “¢A F(,o)>F(, 0), if t}t . 
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the pair (E. p) is a metric space and the pair (£. #) a measur- 
able space, When the c-algebra # is generated by the system 
of all open sets of the space (£.p) we shall write (&. p) for 
brevity instead of (E.p,g). It may be noticed that condi- 
tions 1,9,4 and 1,.9.B are satisfied in this case: p(x, U). 
can be taken as the function f(x), 


5. The Construction of Probability Measures 


1.10 The following general theorem is usually taken as the 
basis for constructing probability measures, 


Theorem 1,1, Let C be a system of subsets of a set E 
satisfying the conditions: 


1.10.4, . EE€@. 


1,10.4,. If AEC, then A can be expressed as the sum of 


a finite number of disjoint elements of @, 
1,10.4,. If A, BE@, then 4Bee. 
Let ®(4) be a function in system C such that: 
1.10.8, . ®(A)>0 a every AEC. 


1,10.B,, If A=U4n, where 4, A, An... €@ and AA,=0 
for ij , then : 


2 2 ®(A,). 
1.10.B;. @(F)=1. 


Then there exists a unique probability measure P(A) on the 
o-algebra o(@) such that 


P(A)= (A) (AEC). (1.26) 


The uniqueness of the measure P follows directly from 
lemma 1.1, For let P be a measure in o(@) which, like P, 
satisfies condition (1.26). We put BE¥ if  B(s)=P(s) . 
Obviously ¥ > @ and is a \-system, Since @ is a x -systen, 
oF Dso(@) by lemma l,l. 


In order to construct at least one measure P satisfying 
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condition (1.26) we put for each BEE 
P(B)= int >) P(A,), 
n=l 


where A,,..., A,,...€ @ and the lower bound runs over all the 
systems {4,} such that UJ A.>B- We consider the system # 
n= 


of all sets B for which P(B)-+P(B)=1. It may be shown 
that the system # contains @ and is a c-algebra and that 
the function P(B), if it is considered only in the s-algebra 
#-, is a measure satisfying conditions (1.26), This is all 
the more true if P is considered only in the s-algebra «o(@), 
which is contained in @, 


1.11 We take an important example, Let E=[a, 5]*) and let 
the function F(uz) (a€[a, 5]) satisfy the following conditions: 


1.11.A.F(u) is non-decreasing; 

1.11.B. F(u) is continuous from the right; 

p ieee GR 8 F (b)-=1 

We write @ for the aggregate of all intervals (s, ?] 
(4S5<t<b) and f[a,t] @gt<b) and put 

G(s, ] =F (@)—F(s), ®[a, t] = F (2). 

It may easily be seen that @ satisfies conditions 1,10,A.- 
1.10.45, and © the conditions 1,10,B,-1.10.5;. The pro- 
bability measure P(A), the existence of which is asserted 
in theorem 1.1, obviously satisfies the condition 


P [a, t] = F (2). (1.27) 


1.12. Theorem 1,2, Let T be an arbitrary set, T a subset 
of T, (EF. @. #) a s-compact topological space satisfying 
condition 1.9.4. We write -? for the class of all functions 
?(4) on the set T with values from E and W» for the s- 
algebra in the space £* generated by the sets {9:9(€T} 
(¢€7,TEC#@) . Let the function®:,...4 (Ti -.-. Tn) (i -... Pn€ #) 
be given for everyn=1,2,... wand any 4,...,4,€7 , and let 
this function be subject to the following requirements: 


*\ a and 6 can take either finite or infinite values, 
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LL L2 A, O32; os ti (Ti ches Y= 91... t, (1, Selae Tn) for any 
1 n 
permitation (i,,.... i,) of the mumbers 1, 2,....”.. 
1.12.B. if the values of all the arguments except one of 
the function %,..,1,( ---» Tn) are fixed in an arbitrary 


manner, the function is a measure with respect to the remain- 
ing argument, 


Lvle cs 0,(AE)=1 3 
Dy, ve bas ty (C1, eee, Py-1, E) = ®,., ee tu cease Tn-1) 
(1 > 1). 
Then there exists the unique probability measure P in the 
c-algebra oy such that, for anyt,,..., EET, Ty... DCB? 
P(O(A)ET, ..., o(fn)ETn} = ,, iyite, (Vissace Ep): (1.28) 


We prove this theorem by observing that the subsets of the 
space f? that have the form 


A={o:e@)Er, +2, OEVET RS 
(n= 1, ae big ara6s eT; T eae T,.€ B) 


form a system C that satisfies conditions 1,10,A)-1.10.A,, 


It may further be verified that the function ®, defined in 
the system @€ by the expression 
® {9 (ty) € Paice o(tn)€ T,} = ®,., es t (Pr eae by 


satisfies requirements 1.10.8) -1.10.5, (the assumptions 


regarding the topology of the space (E, #), introduced in the 
statement of the theorem, are employed in verifying property 
1,10.B,). After this, theorem 1,2 follows from theorem 1.1, 


CHAPTER 2 
MARKOV PROCESSES 
1, The Definition of Markov Process 


2.1 The clearest picture of a Markov process may be got 
from the following description, Suppose a particle is in 
random motion in a space FE during the interval of time [0. 9. 
If the position of the particle is known at the instant t, 
supplementary information regarding the phenomena observed 
up till the instant ¢ (and in particular, regarding the 
nature of the motion until ¢) has no effect on prognosis of 
the motion after the instant i (for a know "present", the 
"future" and the "past" are independent of each other} . The 
instant € at which the motion is cut off may be random, 


The accurate definition is as follows, Given: 


a) a function C(w) in a space @, taking non-negative 
values (including the value -+oo ie 


b) a function <«(t,o)—-x,(w), defined for w€2, FE [0, £(w)) 
and taking values from a measurable space (£, #)(the c-algebra 
# is assumed to contain all the subsets consisting of a 
single point); 


c) for each 0¢s<t. a c-algebra of: in the space 2,— 
{0 (w) > £) 3 

d) for each s>0, x€E a function P,(A) on a o-algebra 
oM® in the space @ containing o#§ for all t>s, 


We shall say that these elements define a Markov process 
X=(x, ©, oft, P, .) if the following conditions are satis- 
fied: 


2,.1l.A.s<t<guand A€o#, then {4,C>u}€ oft. 


2,1.B, (%:ETIEM for any 0<s<t, TE #* 


{C>t} € off} 
for any 0<s<t, 
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2.1.C, P, . is a probability measure on the s-algebra 4s, 
2.1.D, For any 0<¢s<ct, TE ge 
P(s, x; t TY=Ps 2 {%€T} (2.1) 
is a @-measurable function of x, 
2,1,58. Ps. x s,-EN«=0. 
2.1.F. If 0<s<t<u, x€£, TES, then 
Pex [xu€ToM:}=P(t, x54T) (€.c.2 Pz). (2.2) 


The set @ is called the space of elementary events. The 
measurable space (EF, #) is called the phase space, the quantity 
‘ the instant of cut-off (or the life), the function P(s, x; 4, I) 
the transition function of the process X. With » fixed the 


function x,(w) ¢€[0,C(w))) defines the trajectory of the pro- 
cess in the space £ corresponding to the elementary event wo, 


The g-algebra o#; can be pictured as the class of events 
which are observed during the interval of time [s, ft]. The 
value of P, ~(A) (A€ o#*) is interpreted as the probability of 
event A on condition that the particle is at the point x at 
the instant s, 


Condition 2,.1.F can be replaced by the following: 
2.1.F°, If0cs<ct<cu, xCE, AE o#:, then 


Po a(A. Xu€ T= f P(t, x5 2, T)Ps, 2 (de). (2.3) 
A 


For by definition of conditional probability (see sec, 1.6), 
2.1.F is equivalent to the combination of 2.1.F° and the 
requirement that P(t, x, 2, T) be an of -measurable function 
of w, But this latter requirement follows from condition 
2.1.D. In fact, the mapping of ©, into the segment 2=[0, 1] 
defined by the function P(t, x,; 4, T) is the product of the 
measurable mapping of (2:, o#)into(—, #)defined by the func- 
tion x:(w) and the measurable mapping of (£,)in (ft, #) 
defined by the function P(t, x; u, T). 


An important class of Markov processes is provided by those 
for which C(w)=-+oo for all w€2. We shall refer to these 
as non-cut-off processes and write them as (x;, M&M, Ps). 
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Remark, The above definition of Markov processes can be 
somewhat widened, We fix a set T of real numbers and assume 
thats 


a) €(w) can take only values from T and the improper value 


+ 00 5 


b) the function *x;(w) is defined for values of ¢€T less 
than (C(w) 3 


c) the s-algebras of and of’ and the functions P,,, are 
defined only for s, t€7T. 


If this assemblage satisfies conditions 2,1,A-2,.1,F, it is 
said to define a Markov process in the time set T*). On 
putting T=[0, +c), we return to the fundamental case con- 
sidered at the beginning of sec, 2.1, If T coincides with 
the set of all non-negative integers (or positive integers), 
we speak of a discrete Markov process or Markov chain. We 
shall save ourselves unnecessary complexity in the future by 
confining ourselves to the fundamental case T=/[0, +09), 
since the case of an arbitrary set T does not differ from 
this in any essentials, 


Some further concepts related to Markov processes may use- 
fully be introduced. 


We shall describe as a Markov random function in the phase 
space (E, #) and in the time interval /=[a, 5] the aggregate 
of the following objects: 


a) the function €(w) in some set @ with values from the 
segment [a, b] 3 


b) the function +x,(w)=<«(t, w) defined for wE€2,t€fa, C(w)) 
and taking values from £3 


c) for each ¢€/ the s-algebra of, in the space Q,= {> 12}3 


d) the probability measure P on the c-algebra of in space 
Q, containing of, for all t€/. 


It is required here that: 


*)The concept obviously remains meaningful in the case of 
any ordered, or even partially ordered, set 7, 
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2.1. a. {ET} E€o%#, for any TE, tel. 
2.1, m%. o%,[(2,)So#, for any t<cu€/l*. 


2,1. %. P{x,€P]o4,) =P {x,€L]x,} (a.c. 2, P)*) for any 
t<ucs and TES. 


We shall speak of a Markov family of random functions if 
there is associated with each pair s>0. x€ FE a Markov random 


function x" 7 = (x? 7, 6° 7, cM 7, P” “lin the phase space (E, #) 
and in the time interval {[s, oo). The following conditions 
mast be fulfilled here: 


2.1, 8. P(s. x; 4, T)=P”* {x “ET} is a @-measurable function 
of x, 


2.1, Pe Pls, x; s, EN x)=0. 


2.1. By. PY {xh "CLIX? "J]=P x? a D(a.c. OF 7, p*?), for any 
0O<sct<cu, TES. 


Let X=(x;, 6 oft, Ps, x) be an arbitrary Markov process. We 
fix ans>0 and x¢€é& and put 
C& 2 (w) == max (6 (w), 5); 
xf 7 (w) = x, (w) (¢E[s, co), o€ 2), 
My *=ch;  (tE[s, 00)), 
P*7(4)=Ps,2(A) (AE of’). 


The elements (C°%, xf”, of? 7, P* 7) are easily seen to define 
a Markov rendom function in the time interval [s, co), The 
aggregate of these functions for all s>0,xC€E is clearly 
a Markov family. 


On the other hand, a Markov process may readily be associa- 
ted with each Markov family of random functions x* 7 — (x° 7, 
C”*, of? ”, P**), For let @ be the class of triples (s, x, w), 
where s>0, x€E,w€Qsa2, and let 


C(s. x, w) = C% 2(w), 
%,(8, x, ©) = X87 (0), if £E[s, H 2()) 


*) P{—|x, denotes the conditional probability with respect 


to the sc-algebra in the space ©, generated by the seta 
{x,€ BY (BE #) , 
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fwe define the function x,(s, x. ») in an arbitrary manner for 
t€[0, s)]. For each 4c 6 we put 


A® = — fw: (s, x, w)€ A}. 


lie write of for the class of all sets AG& , such that 
A» 7€ of * for any xCE, and of; for the class of all AG @ 
such that A” 7€ oft" for any x€E. It can easily be seen 
that o# isa s-algebra in the space Q.= (T>72} and of? aa- 
algebra in the space 2, where p'>.g# for all t>s. 


~! 


Finally, for every A€o#s we put 
P, (A) = P® *(A® 2). 


_We leave it to the reader to verify that the system (xp ©. of. 
P, 2) Satisfies conditions 2,1,A-2,1,F and is therefore a 
Markov process, 


2.2. We shall use the notation dV} for the c-algebra in the 
space @, generated by the sets iY: x, ()ET)N&, we h, TE B)s 
and o* for the c-algebra in the space generated by the sets 
{o:x,@)ET} Wer, TE). 


In view of conditions 2,1,A and 2,1,B, WN? Goff, and JN’ Gof* 
Obviously, (% © Wt, P,.)is a Markov process as well as 
(x 5, oft, Py a) 


Furthermore, we put A€o#* if there exist sets B, and 8, 


ne for every x€E such that 8,2 AGB, and P, ,(B,) =P, 
w\"2) « 


On putting Py, 2(A)= Pe, 2 (Bi) =Ps, o (Bs), we extend the pro- 
bability measures P,,, to the c-algebra of, This evidently 
does not affect the fact of (x,, 6 of, Ps.) being a Markov pro- 
cess; it is not affected even by replacing the s-algebras 
ot by the wider c-algebras of], which are constructed as 
follows: we putACo#fif AG of’, AG; and for every x€E 
there exists 5 EC of; Such that 


P,,2(A\ AB) =P,, 2(B \ AB) =0. 


Theorem 2,1, Let (% 0 of: P, .) be a Markov process and 
let O<¢sect . If BEW* , then 


P,, o(BloM) =Py, », (B) (a0 2, Py) ")- (2.4) 


*)It is clear from (2.4) that, for any Bed the function 


P,,¥(B) can be constructed in accordance with Py. as far as 
a y-set I’ , for which P(0, 4% 5, I’)=0. , 
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If t is W’ measurable and P, .~Summable, then 


Me, 2 JoM:) = M,, x,* (a.c, 2, Ps, 2) (2, 5) 


Proof, Expressions (2,4) and (2.5) will be proved simultan- 
eously, We start by showing that (2.5) is satisfied for 
=f (x,), where u>?t and f is a bounded #-measurable func- 
tion in space F. Let - be the class of all bounded func~ 
tions f(x) (x€E) and $ the class of all #-measurable 
functions f such that f(x.) satisfies (2.5). Clearly, # is 
an ./-system, In view of 2,1.F, 4 contains the characteris- 
tic functions of all sets TE @. By lemma 1.2, & contains 
all bounded @-measurable functions, 


We now prove that (2.4) is satisfied for every set 


B= (xu, ED; -. tu, ET a} CS ey ee Mei Tyo. ThE #). (2.6) 


If n=1, this is true in view of 2.1.F. We now use induc- 
tion, let 


B,={xu,ET,}; Bo (x,€Te ..-. xu,€T}- 
Obviously, p—p,B, » and by 1.6.H, 
Pa, (BY oft) = Ma, 2 {Xp,Ps, 2(B2loMuloMi} (2.0 +2; Py, 2). 
We have by induction: 


P,. zx (B, | Me, ) — Pu,, wy, (Bo) (a ela Qa P, 2): 
Therefore 
P, x (BI oM;) a M, Ks {43 Pu. T1, (Bz) i oN} 7 
= Ma, z {f (Xu,)] M;} (A.C. 2 Ps, x)s 
where 


F(X) = 7p (2)P, (By). 


On applying the particular case of (2.5) already proved, we 
have 


1. Pa {f (Xu,) [o%:} = Mz, a, f (Xu,) (a.c, Q,, Py, 2): (2.7) 


On the other hand, by 1.6.H and our inductive assumption, 
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P;, y(B)= Mz, y ‘Lp,Ps, 7 [Bz fcKs,]} = 
= M, y [7_3P ts Ly, (B,)] =— My yf (x,,). (2 8) 


On comparing (2.7) and (2.8), we conclude that (2.4) is 
satisfied for all sets of the type (2.6). 


We now write .& for the class of all Ps,z-summable functions 
E(w) (w0€2). Obviously, the set Sv of all functions for which 
condition (2.5) is satisfied is an %-system, By what has 
been proved, 3 contains the characteristic functions of all 
sets (2.6). The latter form a <-system generating the c- 
algebra Nt, By lemma 1.2, <@ contains all .\t -measurable 
P, 2 -Summable functions, Relationship (2.5) is now fully 
proved, If Bis any set of N*, on putting t= 7, in (2.5) 
we get (2.4). 


Corollary. If A€ch#;, BEN , then 


Ps, » (AB) = i) Pr, 2(B) Ps, » (de). (2.9) 


If ¢ is Wi:-measurable 7 o‘-~measurable and £y P, »-summable 
ther 


Mg, 2} = M, a (EMy, «,7] *). (2 10) 


Expression (2.9) follows from a comparison of (2.4) and 
(1.11), and (2.10) from a comparison of (2.5) and 1.6.1. 


2.3. We put 
P(s, x; A; t,.T)==P. [4,4 ET} 
(O<s<t, AE of; TE #). 
On the basis of lemma 1.6, we can write condition 2,1.F° in 
the following forn, 
2.1.F°°, 0<s<t<u,TE€#. ACM, then 
P(s, x; Ara, Ti = [P(s, x: Art dy) PQ ys wD). (2,11) 


K 


*)The functions 2 and EM. 2,7 are defined only on the set 2, 


In accordance with sec, 1.6, their mathematical expectations 


are understood to imply integration over the set &,. 
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We notice that, with 4=2, P(s,x; Aria. T)=P(s. x; 4,1) and 
equation (2,11) becomes 


P(s, Me ees rT) = [Po. Baa dy) P(, ¥; U, Y) (2 12) 


E 
O¢scr<cu, TC #). 


This relationship for the transition function of the pro- 
cess is usually known as the Kolmogorov-Chepmen equation *), 


We put 
P(s, x; ty Fis od fis 1) =P, ao (Xt, El’, weer aod x1 € r,) 
2.13) 
(OE OS 82 hy cian t Teele @: ( 
Let sct<c...<t,. On putting 
A= (Xe, c Ty eae oe, ET aa}  ._—— a ee Fi) =th, PT, 
in relationship (2.11), we get 
Pi. trip Ty... 8, 0) = 
= : PS 0s ts Died aig by grtty PC as Hels). (2.14) 


Ty —1{ 


Hence we find by induction that 


P(s, x, fy Ty ..oty T,) = f- a f Pos. x3 ty, dy) X 
MY Yn-1 (2,15) 
XP (ty M5 fe dye). .- Ply Yaw fo Ty) 
(sich... Ct Ty... Ta€ #). 


Lemma 2,1, If of#?= WN? , condition 2,1.F in the definition 
of Markov process can be replaced by the requirement that 
(2.14) be satisfied for any n=1,2,...,.0<s<ct,che... 
<t, Ty... Te. 


Proof, It follows at once from (2.14) that condition 
2.1.F°’ is satisfied for any 
A=({Q, x1, ET... x, ET} 
(a==1,2,...34, ...,8,E[s, f]. (2.16) 
T,,.. 5 T,€ F). 
*)On putting in particular !'=£, s=t=u and taking 2.1.5 


into account, P(s, +: 5.2) is seen to equal 0 or 1 for any 
$24, xCE& 
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Sets (2.16) form a x-system @ in ©,, On the other hand, 
the class of all sets A for which condition 2,1.F’’ is 
satisfied is a \-system (in ©,), By lemma l,l, since # > @ 
we have ¥26(@)=aWN; , and the lemma is proved, 


Lemma 2,2, For any s>0, AGW P, (A) is a #-measurable 
function of x, 


Proof. We show by induction over n that all the functions 
P(s, x; t, Ty ....4,,T,) are measurable with respect to x, Our 
assertion follows for »=1 from condition 2.1.D, The passage 
from n—I1 ton is accomplished by means of lemma 1.7 and 
equation (2.14). 


We now remark that the class ¥ of all AE€W* for which 
the assertion of the lemma holds is a )h-system, By what has 
been proved, contains the n-system @, consisting of the 
sets 


A= {xz ET), eee x2,€ r 
(tale Qinkxs ti ice ts eS 
YT; ees T,€ #). 


In view of lemma 1.1, ¥20(@)=WN*. The lemma is proved, 


2.4. Let » be an arbitrary measure on the c-algebra @, In 
accordance with lemma 2,2, the integral 


Psp (A) = f Po, o(A)p (2x). 
E 


has a meaning for any A€ df. 
It may readily be seen that this integral defines a measure 
on the c-algebra oN and that (2.3) [and therefore also (2,2)] 


remains true on replacing P, , by Ps,.*). If» is a proba- 
bility measure, Ps,» is also a probability measure, and the 


*)It may easily be verified, on the basis of lemma 1.2, that 
ffre Py, x (do) (dx) = fF) P,, wu (do). 
Eg : 


for any W” -measurable bounded function f(w) . Expression 
(2.3) soon follows from this, 
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value P; (A) is naturally interpreted as the probability of 
event A if the moving particle has a probability distribu- 
tion p at the instant s. 


We write ACW if, for every measure » on the s-algebra 
@, an Aj, A, of f° can be constructed such that A;C ACA, 
and =P, »(A\)=P,,.(A) . Obviously, AW’co#* . The measure 
P,,, can be extended to the c-algebra ,f’ in precisely the 
way the measuresP,;,z were extended in sec. 2.2 to the c- 
algebra of', 


Theorem 2,1°, Expressions (2,4) and (2.5) remain valid fo 
any BEd and any d’-measurable P,,.-summable function £, 
Expressions (2,9) and (2,10) remain valid for any A€oMi, , 
BEN’ and any functions =, 7, such that is of; -medsurab] 
7 oft-measurable, and y-and & Ps, , -summable*), 

Proof, let A€ocH#:, BEN. The expression 

eT) = Ps 2(A, ET) TE #) 


defines a Measure in #,. We choose 8, and B, from Wf * such 


that8, [82 Band Pp, .(B,) =P: .(B,)- By the corollary of 
theorem 2,1, 


Ps, x (AB;) = f Py, x, (B;) Pa, x (dw) = 
A 


= f » (dy) Pr, y (B:) = Py, » (B,). 
E 


It follows from AB, c ABC AB, that 
Ps, .(B:)=P, , (AB) < Py, 2 (AB) < Ps, x (ABl) = Pz, » (By). 
The extreme terms of this inequality are equal , so that 
Pe, 2(AB) = Py, (Bi) = f Pt, 2, (Bj) Ps, 2 (do). (2.16’) 
A 
Furthermore, it follows fromB,c Bc B, that, for all w€Q, : 
Pt, 2, (Bi) < Pe, 2, (B) < Pr, x, (Be). 


The integrals of the extreme terms over set A with respect to 


*)We assume that all the measures P,. are extended to the 


o~algebra of® as described in sec, 2.2. 


MARKOV PROCESSES 35 


the measure Py;,z are equal, therefore 
Py, oy (B,) = P, x, (B) = Pz, a, (B,) (a,c, A, Pas) 


Comparison of this expression with equation (2,16’) shows 
that (2.9) is satisfied. Hence it follows, by the definition 
of conditional probability, that equation (2.4) holds for 
every BEd. We deduce (2.5) from (2.4) by making standard 
use of lemma 1.2, Finally,(2.10), follows from a comparison 
of (2.5) and 1.6.3. 


Remark, As mentioned in sec, 2,2, the Markov nature of a 
process is not affected by replacing c#% by of®. We can 
therefore write .43 for c#; in the statements of theorems 
2.1 and 2,1’, 


2, Stationary Markov Processes 


2.5. Let o\'* denote the minimal system of subsets of the space 
Q,=(C>0} that contains all the sets {x,€T} ¢>0, TE #) 
and is closed with respect to the operations of addition and 
intersection of any number of sets and with respect to the 
operation of taking complements, The Markov process X= (x,, ¢, 
oM:, P, -) is said to be stationary if a subset #AGQ can 
be associated with any ¢>0 and any subset A¢€,i?* such that 
the following conditions are fulfilled: 


2.5-A. §,2,=2, 0,(A\B)=8,A\ 4,8; 


6 (J4.) = U0.4.; (14) =10,4, 


(the index a runs through an arbitrary set of values.) 
2.5.B, 4 {x, ET} = (414,€T} (A>, TE F). 
2.5.0, For any A€ A = oN'[Q] P, 2 (8,4) = Po 2 (A). 


It may be observed that., if operators 6, and 6, satisfy 
conditions 2,5,4-2.5.C, the system of sets uf for which 
6,A—6,A contains the sets {x,€I} (A>0, T€ g) and is 
invariant with respect to all set operations. It follows 
from this that operators 6, are defined uniquely for the 
process xX, The following properties of the 0, are readily 
derived from 2,5,A-2.5.C: 
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2.5.D, Os, = Ven 


2.5.E. If Bead, then 0,8¢ qf and P,,,(0,8)=P, .(B)(we 
assume that the measures P, , are extended to A, 28 indica- 
ted in sec, 2.2, and put BE WN if BEW« and for every 
measure ‘tv in # there are sets 8, and B, of WN such that 
B,S BSB, and Po ,(B,)= Po, » (Bg). » 


2.6. Let the function §(w) (~E2y) be jf*-measurable, The 
sets {&(»)=a} are mtually disjoint and form the sum Q@,, The 
sets 6,{(w) =a} are therefore also mutually disjoint and form 
the sum 0,2) = Q, - We put 06 (m) =a if w€0,{§=a! . Hence 
for every wN’*-measurable function §(w) (w€&,) there is an 
associated function 6,§(w) defined on the set Q,, 


We can readily deduce the following properties of operators 
8. 


2.6.4. For any numerical set [: 
BST} = (8 ET}. 


(it is clear from this that the function 0,@) is  O\"* - 
measurable, ) 


2.6.B, The necessary and sufficient condition for 7=6; 
is that, for any a, 


or that for any a, 
8: (§ >a} = {n> a}. 


ae 6f=C—t (wEQ,), (This follows from 2,5.B and 
2. 3B @ 


2.6 5D, a= Xen (BE oV"*). 
2,6.B, If f(%..., x, ---) is any function in the space R® 


and &%,...,5,... are Seninary '*-neasurable w-functions, 
we have 


Def Ere «22 av SOR -- 02 Offa od. 


In particular, the operators 6, preserve all the algebraic 
operations and the operation of passage to the limit. 
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2.6.F. For every o’-measurable function é: 
Mz, 2943 = Mo, of (2.17) 


The definition of operator 6, as also properties 2.6.4, 
2.6.D, 2.6.E and the first half of 2.6.B are readily extended 
to functions whose values, instead of being numerical, are 
points of any measurable space (F, #). (The main thing is 
simply that # should contain all sets consisting of a single 
point.) The functions (wu) can in this case be defined in 
some subset ©2¢€,\** instead of throughout the set &, ( 0,3 
being now defined in 98). If follows from 2,5,B and 2.6.4 
that 


2.6 os G4%), = Xp 27: 


Theorem 2,2, Let (x, %, of}, P,..) be a stationary Markov pro- 
cess, Let O<sctr. If Bed’, then 


Po x (6,8 | oti) = Po, , (8) (2.C.Q,, Po, ae (2 18) 
If — is cf’-measurable and Po ,~-Summable, then 
Mo, 2 (8¢:fo#t) = Moz @.c.2y Po, 2). (2.19) 
This is proved simply by comparing theorem 2,1° with 
expressions 2.5.E and 2,6.F. 


Corollary. If A€of#;, BEd, then 


Po, 2 (A,B) = f Po, x, (B) Po, « (dw). (2,20) 
A 


If § is of#-measurable, xyof-measurable and 7 and [6,7 Po. - 
summable, then 


Mo, a (867) = Mo, a (5Mo, «,n)- (2:20") 


2.7. We have already seen in sec.2,5 that the operators 6, 
are uniquely defined by the process xX, ‘The conditions will 
now be discussed in which we can assert the existence of 
such operators 6,, i.e, stationarity of the process x —(x,,(, 


A}, P,, x) - 


Theorem 2,3. The necessary and sufficient condition for 
the Markov process X=(x, 6 of, P,.2) to be stationary is that 
it satisfy the following conditions: 
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2.7.4, Pla, x,t +h, T)= PO, x; t, 1). 

2.7.B. For any 0<i<C(w) there exists w’€Q, such that 
2.7.3). C (w’) = C (w) —F. 

2.7.B,. X1,,(0) =x, (0) for all 0¢k <l(w’) = E(w) —?. 
Suppose that the process xX is stationary. 


We shall write © =—c;(w) if w’ and » are connected by 
relationships 2.7 Bl -2.7.B,. Now 


9As={w:cwE A}  (AEdN*); (2,21) 


845 (w) = 8 (cy) (2,22) 
(: is an W*-measurable function, w€Q).) 


Proof, Necessity. Condition 2.7.4 follows from 2.5.B and 
2.5.C. We show that condition 2.7.B holds, For every w€Q, 
let A, be the intersection of all sets A€oN" such that 
w€0A , Obviously, 4,€oN" and w€6,A,. We notice that, 
if w’€a, , conditions 2,7 oB,-2.7 By are satisfied, In 


fact, let “(w)=4, *4;,(w)=a. Then 


w€ (Sa, x14, 4} =0,{.=u—t, x=). 
Therefore w’'€ A, S {2 =u—t, x,=a} and {(w’)=u—s, x, (v’)=a- 


Sufficiency, We show first of all that, if A€WN"' , either 
all or else none of the values of ©” belong to A. The class 
of all sets A having the property in question is in fact 
closed with respect to all set operations and contains (by 
2.7.B,-2.7.B,) the sets {x,€T} (h>0,TE€#). The aggregate 
thus Tontains oN", 


It remains to show that condition 2.5.C holds. Let be 
the system of all sets A for which 2.5.C is satisfied, Let 
C denote the system of all sets 

A= {xn€Ty .... xn,€Tr] OCha<... <h,). (2.23) 


Obviously, ¥ is a }-system and @ a z-system in the space 
Q, e 


Moreover, for sets (2.23), we have 
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Po, 2 (A) = PO, Xx; hy, we oo ee hh, Y,)» 
Pr» (O¢4) = P(t x3 FER, Py eee t-E Ry Bd. 


We observe on comparing 2,7.A and (2.15) that Po2(4)=—= 
Py. 2 (8,4). Therefore ¥ 2@ and by lemma 1.1, ¥ 33(@)= 


NW [Qo] - 
Operators 6, thus satisfy all the requirements 2,5,A-2.5.C, 
Relationship (2.22) follows from (2,21) and 2.6.D. 
9.8.Let (x; 6, o&}, P,, 2) be a stationary Markov process and let 
§, be operators satisfying conditions 2,5,A-2.5.C. We con- 
sider the system (x,,{, of, P,,.6,),, where .of,—of?: P,—=Pos. 
Evidently this system has the following properties: 
2.8.4. If t<cu and A€o#,, then [A, 5 >a} € of, 
2.8.3. {x,€T}€o%, ¢>0,TE #) 
2.8.C, P, is a probability measure on the s-algebra of. 
2.8.D. Forany t>0.SE@: 
Pt, x, T)=P,{x,€T} 
is a #-measurable function of x, 
2.8.8, P(0, x, EN x)=0. 
2.8.F. For any ‘>0, ACW 
P., {6,A] of} = Px, (A) (a.c.Q,, P.). 
2.8.6. 4,2, = 2), 0,(A\ B)=6,A\ 8,8, 
0, { U A,} =U On4q: 0, { N A,} =f} 0,4, 
(2 runs through an arbitrary set of values). 


2.8.H. %{x,€T}={x,.,E€T} ¢>0,4>0,TE #). 


Theorem 2,4, Given: 
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a) a function C(w) (w¢@@) taking values from the interval 
[9, + co]5 


b) a function ~,() defined for w€2, #€[0,C(w)) and taking 
values from the measurable space (E£, @) 5 


c) for any t>0 a c-algebra of, in the space Q,= {C>4#}; 


d) for each x€E a function P(A) in a c-algebra of in 
the space 2 containing of, for all ¢>03 


e) for each ¢>0 and A€AN* aset 4ASQ2 (the system 
NWN is defined in accordance with x,(w) and © as in sec, 2.5), 


We suppose that the system (x; 5, of, P,, 6,) satisfies condi- 
tions 2.8,4-2.8.H. Then there exists a stationary Markov 
process X=(x1, 5, oft, Pc.) for which of;=of, Prp=Poz 
and 6; is a system of operators satisfying requirements 
2.5.A-2.5.C. 


Proof, We put AM; = M,, Po2=P.. With s>0, let of* 
denote the class of all sets of the form 0,4 and 2,U 0,4 » 
where A€N° , and let 


Ps, 2 (8,4) = Pz (A), 


P,, 2 (Q_U 9,4) = 1 — P,, (Qo) + Pz (A), 


oM; = O,N,_,"). 


It may easily be verified that the system (x,,{, of%, P, ,) 
possesses all the properties 2.1.A-2.1.F, whilst the opera- 
tors 6, possess all the properties 2.5.4-2.5.C. The only 
property whose proof requires a certain amount of working is 
ape Re 


To start with, a function 0, can easily be defined for 
every df*-measurable function as in sec, 2.6. Properties 
2.6,.A-2.6.G are soon seen to hold here. 


We have to verify that, for any s<ct<cu, TE, ACK, 
Pen {Ar Xu € Pr} = f Pe x, 2, T) Py, 2 (dw) = oor 
A . * 
= M, 214 4P x,; 4, T)]. 


*)N, denotes the :-algebra in %={£>/} generated by the 
sets (uel, >t (O<u<t Leg), and We the s-algebra in 2 
generated by the sets {x,€!'} (uO, Teg). 
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This relationship is obvious if s=0. If s>0, we have 
by definition of of:A=—6,B, where BEod#,,. By 2.8.G, 2.8.H, 
2.6.D and 2.6.5, 


6, {B, x,_,€T} = iA, Xn€ ine 
O[xpP (4 <3 4 D)) = 4P (6%, 4. T) 


and by 2.5.C and (2.17), relationship (2.24) is equivalent to 
the equation 


P2 (5, *,_,€T} = MiXpP (f %,_,3 4 r= 
= f P(t. x94 T)P, (do). (2.25) 
B 


To prove (2.25), we only need to verify that 
P, {~,-eE TJ o%,_,} =Ptt, X45: 4, T), 


But, by 2.8.F and 2.8.9, 


P, {X45 S Tf o%;_.) =P, {9,_. [Xy_-2€ Df o%,_.] = 
=Pa,_.{*u-+ET} =P (u—t, x,_,, T)(a.c.Q,_,, P,). 


It remains to observe that, by 2.5.0 and 2.8.H, for every 
YEE 
P(t, y; u, DY) = Pry (%u€T} = Py (*,-1ET) =P(u—té, y, I). 


The theorem is thus proved, 


We shall employ the term 'stationary Markov process" below 
in two different senses, either for the system (x, C, ofj, P,, .) 
subject to conditions 2.1,A-2,1.F, for which a system of 
operators 6, exists satisfying requirements 2.5,A-2,5.C, or 
for the system ‘x,, C, of, P,, 6,) subject to conditions 2,8 .A- 
2.8.H. Writing X=(x,C, 04, P,,,) denotes a stationary Markov 
process in the first sense, whilst x’ =(%; 6 of, P,, 6,) denotes 
a stationary Markov process in the second sense, If %#=o#,, 
Po,2==P, and operators 9, satisfy conditions 2.5,A-2.5.C in 
regard to the process xX, we shall say that ¥ corresponds to 
X’ (or X’ corresponds to X) and write X<~>.x’. For every 
stationary Markov process in the first sense there is a 
corresponding stationary Markov process in the second sense, 
The converse is true by theorem 2,4: for every stationary 
Markov process in the second sense there is a corresponding 
stationary Markov process in the first sense, The two 
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meanings of the term "stationary Markov process" are thus 
very closely related, The difference between them amounts 
to this: when it is a question of a stationary Markov pro- 
cess in the first sense, our knowledge of *t (, off, Py x 

is essentially complete, whereas when it is a question of a 
stationary Markov process in the second sense our knowledge 
of x, 6, Ai, Po, 2 6 is essentially complete but there is some 
arbitrariness in the choice of c-algebras of; and measures 
P,.2for <s>0. 


3. Equivalent Markov Processes 


2.9.Let X==(x, 6 of}, Pe, 2) and X=(x, © off, Ps, «) be two Markov 
processes in the same phase space (F£, #). We shall say that 
X is got from X by means of the transformation of the space 
of elementary events 7:82 (Q! is the space of elementary 
events for x, 9 the similar space for ¥, and 7 the mapping 
of Q into 2), if the following requirements are satisfied: 


2.9.4. S(o)=CI7(o)] WE). 

2.9.3. “(=x [7@] @ER0 ct <lO)=CIy op. 
2.9.C. i= 1-'(oMi) *)- 

2.9.D. of’=y-'(o#') and P,,(7-14)=P,2(A) (AC of’). 


Evidently the process X is uniquely defined, given the 
process X, by the transformation y, Now what conditions 
must be imposed on the mapping y+ for a Markov process satis- 
fying conditions 2,9,.A-2.9.D to exist? The answer to this 
question is supplied by the following theorem, 


Theorem 2,5. The necessary and sufficient condition for a 
mapping 7:2 4Q to give a transformation of the space of 
elementary events of a process X is that: 


2.9.0. For any s>0, x«€E, A€oM#’, it follows from 43 +(8) 
that P.2(4)=1**). 


*);-1(A) denotes the complete pre-image of A for the mapping 
tT, i.e. the set {wry(aje A}. If # is a system of subsets of 
the space Q,7-1(4) dénotes the system of all sets 171(A) (Aéc4). 
**)In other words, no matter what measures P,,, we start 
from, the corresponding outer measure of the set 7(@) is 
equal to unity. 
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Proof, If A27(2) , then 7-'(A)=9 and by expression 
2.9: 9D 1 P, ~(A) =P, ,(8)=1.. We have thus proved the necessity 
of the condition, Turning to the proof of sufficiency, we 
consider the elements {, Xi Hi, of and Py. 2 defined by expres- 
sions 2.9.4-2.9.D. We show first of all that the function 
P, ,(B)is defined uniquely for any B€o#*, This is done 
simply by verifying that, if 


B= 1-1(A)) = 171 (4), (2.26) 


then 
P, (A) = Ps (Ao). 


In fact, it follows from (2,26) that 
1? (At\ Ars) = 17 (ANN 17 (AD 1 (A) = @ 


Consequently A,\A,;A,27(8) and by 2.9.9, Ps2(A\AA)=1 
and therefore Ps,2(Ai\A142)=0, Ps 2 (A,) =P; a(ArA)+ Tt may 
similarly be shown that P, (Ao) = Ps, (AA) 


The proof of the fact that the system (xp ia of, Py. -) 
satisfies conditions 2,1.A-2.1.E and 2.1.F” presents no 
difficulties and is left to the reader. 


We shall take two particular cases of transormation of the 
space of elementary events, 


a) If y is a one-to-one mapping of @ into 2, 9 can be 
identified with some subset of set @&, The passage from 
process ‘X to process X reduces in this case to restricting 
the domains of definition of functions {(w) and x,(w) , to 
replacing each set A of &M®(or of of) by its intersection with 
Oand to a natural transfer of measures P, 2 to these inter- 
sections. We shall speak in this special case of the pro- 
cess X being got from X by a restriction of the space of 
elementary events. Given X, the process ¥ is uniquely 
defined by the set ©. By theorem 2.5, this latter is sub- 
ject to the single requirement: 


2.9.a°. For any s>0, x€E, A€ok, it follows from AD® 
that P, 2(A)=1 . 


b) If y is the mapping of © into @, each point »€@ can 
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be visualized as being "split" into the set of points _ 
+-1(w)=(0:7(0)=0}, ‘In this case we shall say that X is 


got from X by splitting the elementary events, 


It may readily be seen that a general transformation of the 
space of elementary events can be got by successively purging 
the space of elementary events (when @ is restricted to7 (&)) 
and ae the elementary events (signifying passage from 
1 (&) to @). 


2.10. Let X= (Xp, C At:, P, 2) and x= (xy, C; of}, P, a) be two 
Markov processes in the same phase space and with the same 
space of elementary events, and let 


2 ~L0 Pes os = M3. 
2.10.B. of ao#® and Pz, ,(A)= Py, (A) (AE of). 
We shall speak here of 


(or of X being got from 
algebras, 


being got from X by a widening 


x 
x, by a narrowing) of the basic s- 


The operation of supplementing the measures described in 
sec, 2.2 can be quoted as an example of widening the basic 
¢~algebras, As an example of narrowing these algebras, we 
may mention the passage from‘s-algebras off, o#° to c-algebras 
3, W*, Which was also discussed in sec, 2.2, 


2.11.We shall speak of the Markov process X being subordin- 
ate to the Markov process X if X can be got from X by 
carrying out successively a transformation of the space of 
elementary events and a widening of the basic so-algebras. 
The necessary and sufficient condition for this is that some 
mapping +:0-,@ satisfy the following requirements 


2.11.4, (o)=Cy@)] @E®) 

2.11.3. %,(0) =x, [7 (0)] ES, 0<t< [(a)). 

2.11.C, oft Dy-1 (cA}). 

2,11.D. of’ Sy-1(oM*) and B, ,(7-14)=P,, , (A)(AE of). 

If process X is subordinate to process X, every traject- 


ory of X is at the same time a trajectory of xX, The trans- 
ition functions of processes ¥ and xX are the same, 
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We shall consider every possible interval [0,4) and every 
possible function ¢?() defined in these intervals with values 
in the space E, The class of all these functions will be 
denoted by @,. let ¢€2,; . We shall write ¢(9) for the 
right~hand end of the interval in which the function ? is 
given, Two elements ¢ and ¢ of &, will be reckoned identical 
when and only when {(¢)= fi») and g()=¢(f) for all 


0<t<li@=Cm) *). 

We put ~(7)=9() O<t<%(y)) and write WN” for the o- 
algebra in space ‘2; generated by the sets {9: x,(~)€T} 
(a>s, TE gw) and AN; for the s-algebra in space {y. C(o) > ths 
generated by the sets [7:x,(9)ET, C(e)>t} (Els, th, PE @) » 
The Markov process (x, 5, o#:, P,,.) in the phase space (f, g) 
will be said to be canonical if we have for the process 


2Q=k,, Xp == Xp, roe, Mi = Ns : A> = N*. 
Lemma 2,3, Every Markov process X=(x,,l, of3,P, ,) is sub- 
ordinate to some canonical process ¥ — (x, r Ms, p y° 
> ’ §,.2z 


Proof, Let @ be the space of elementary events of the 
process X, We associate each »€2 with a function ?(4 
having values from EF and defined in the interval [0. [(w)) 
by the expression 


eO=x (a) (O<t <C(W)). (2.27) 


Expression (2.27) defines the mapping 7:22, **), 
We evidently have here 


CL7 (w)] = % (w), 
x, [7 (w) ] = x; (), 
171M") S of*, 1-1 (NS) SoM. 


For every A€.\* we put 
Ps, 2(A)= Py, 2 (1-14). 
The system (x, C AN%, B, ,) satisfies conditions 2.1,A-2.1.F 


*)One of the elements of the space 2, is the function ¢ 


which is nowhere defined, For this, (€(,.) => 
**)If F(w) =0 » we put 1M=% y where % is the element of 
*z defined in the previous foot-note. 
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and therefore yields some Markov process xX . This process is 
canonical and the process X is subordinate to it. 


The canonical process X constructed in lemma 2.3 will be 
called the canonical form of the process X, 


Remark, All the concepts and results given in sec, 2,9- 
2,11 for Markov processes may be carried over directly to 
Markov random functions, In particular, we can assert the 
following, which will be useful later, 


The necessary and sufficient condition for the space of 
elementary events of the Markov random function X= = (Xp % n hy 
P) to be reducible by restriction to the subset G=qQ is 
that, for every 1¢4, it follows from 4yz>Q that 
P(A) =1. 


2.12.We shall speak of the Markov processes XxX’ and X” being 
equivalent if they are defined in the same phase space and 
have the same transition function, 


Theorem 2,6, Every class of equivalent Markov processes 
contains one and only one canonical process X and consists 
of all the processes subordinate to xX, 


Proof, By lemma 2,5, it is a matter of proving that, if 
two canonical processes x’ =(x, 0, AN, P,,,) and NG, é, 
N's, Ps. z) have the same transition functions they mst coincide, 
We thus need to show that, for any A€ WwW", 


Pi (A) =P? , (A). (2,28) 


By (2.15), this relationship is satisfied at least for the 
sets 


= {ET -. 6) ty, ET a} 
(G1) 2 et Cs eet. Dioiag Phe @): (2.29) 


The sets for which equation (2.28) is satisfied form a )\- 
system ¥, whilst the sets of form (2.29) form a ™-system @, 
By lemma 1, 1, it follows from ¥ >C that ¥ Do(@)=—N". 


We shall discuss in detail in Chapter 4 the question of 
what sort of functions are transition functions for Markov 
processes, 
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2.13.Let LY be a subset of the set Q.. Let py denote the 
aggregate of all Markov processes corresponding to the transi- 
tion function P(s, x; t, T) and such that all their trajectories 
belong to the set .&, The aggregate c% p.y may be empty. If 
it is non-empty, we can conclude on the basis of theorem 2.5 
that: 


a) the set 2 satisfies condition 2.9.a with respect to 
the corresponding transition function P(s, x; ¢, [) for the 
canonical process X 3 


b) the space of elementary events of the process X can be 
contracted by means of a restriction to the set 2, 


_We shall describe the Markov process obtained as a result 
of such a purge as -canonical, The following variant of 
theorem 2.6 admits of a simple proof which we shall leave to 
the reader. 


Theorem 2,7. If the class:#p,¢ is non-empty, it contains 
one and only one -#-canonical Markov process X and consists 
of all the Markov processes subordinate to xX, 


In particular, on choosing as 2 the class E’ of all 
functions defined in the interval J=[0, 00), we arrive at 
the following result, 


Corollary. Every class of equivalent non~cut-off Markov 
processes contains one and only one canonical non-cut-off 
process X and consists of all non-cut-off Markov processes 
subordinate to X, 


We shall now prove an important theorem which enables us 
to judge whether some class of equivalent non-cut-off Markov 
processes contains a process whose trajectories possess cer- 
tain previously assigned properties, 


Theorem 2,8, Let X=(x, cM, Ps,z) be a non-cut-off*) 
Markov process in the phase space (£, @) and let .%¢ £F! 
We shall assume the existence of a non-negative function 


Qik Kis cc byi eye as) 
(O< ft, Reel ae ees Ky eet ese Xn eee € E), 


*) A similar theorem for cut-off processes is proved in 


sec. 6.1. 
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which, for any s>0 and for any denumerable everywhere dense 
subset {t,....¢,....} of the interval [s. co) isa #°~ 
measurable function of *; --- x,,..., and satisfies the follow- 
ing conditions: 


2.13.4. If g(t x, ...,fg Xp...) = 0, there exists oC _— 
such that 9 (¢,)=x, (A=1,2,...,a,...). 

2.13.B. Mg, OF (brs Xt. «+> Ear Xyys ++) 0 (or what amounts 
to the same thing, Ps 2 {G(bi Xp. ---0 bn Xp, ---) KO} = 0). 


Then there exists for the process X an equivalent Markov 
process, all the trajectories of which belong to the set 2, 


Proof. We shall take the canonical form of the non-cut- 
off process X: X=(x;, Wt, P, ,). The theorem will be proved 
if we can show that condition 2.9.a is satisfied for process 
X and set , i.e, that it follows from AD_Y thatP, ,(A4)=1 
no matter what s>0, x€E and A€ *. 


Thus let A€c* and AD.Y, On the basis of lemma 1,5, 
we can write the characteristic function of A in the form 


La (=F Xe, (Ps 0s He (Ms DS 


2.50 
= f(t) »--1 On) «+ +] (PEE): (2.30) 
where f;,...,f,... is some sequence of points of [S, oo), 
We can obviously suppose without loss of generality that 


fi, ..+ +f), ... is everywhere dense in [s, co), 
We show that, if CA , then 
Gti, B(tD, «+0. tar V(t)» «--) #0. 
If in fact we had 
Qty V(ty). 0s Ege D (Ey) «J =O, 
there would exist a function ¢€.% by 2.13.4 such that 
V(t.) = 9(t,) fork=1,...,0,...By (2.30), 4.) =x,(9) , and 


?€A inasmuch as ¢€.7S5A ., It now remains for us to 
observe that, in view of the equivalence of X and Xx, 


My 2 9 (tis Xt,s eeeys bis Ves ee -)= Mg, 29 (by Xtys eves L ad e .) (2.31) 
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(this is proved by conventional use of lemma 1.2), 


It follows from (2.31), 2.13.B and the fact that the func- 
tion 7g is non-negative, that 


~ 


Ps 2 {9h Xt, weep bay Xt pea) Od 


so that all the more, 


P, 2(A)=1. 


Remark, Let all the trajectories of the non-cut-off pro- 
cess X belong to the set .%),S EF! and let the function 
Q(ty, %1,..++ by Xp, ...) Satisfy 2.13.B and the following 
conditions: 


2.13.A°. If b€.f% and (41,46), -..,t,, P(t,), ...)=0 , the 
function 9€.7N.% exists such that 9 (f,)=(¢,). 


Then there exists an equivalent of X, all the trajectories 
of which belong to YN.%.. 


In fact, let X be the canonical form of process X., Let 
AGW and ADPNS, . We put p68 if We bG). ... ty 
b(ty),...)#0. We show, as in the proof of theorem 2,8, that 
PNASB Therefore .%)SAYB and by theorem 2,5, 

P, .(AUB)=1-° By condition 2.13.B, Ps,2(B)=0 , so that 
P., (A) = 1. 


2.14. We shall give an illustration of the application of 
theorem 2.8, 


Let T and G be measurable sets in the space (£.%), We 
shall speak of T being inaccessible from Gif, for any 


0st 
(%_(w) € G} & (x, (w) ET. 


If set T is inaccessible from ENI', we shall simply say 
that Y is inaccessible, 


Lema 2,4, If the transition function of a non-cut-off 
process X satisfies the condition 


2.14.4. P(s, x; 4, M=—0 for all x«€G, 0csct , a process 
equivalent to X exists for which the set [ is inaccessible 
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from G, 


Proof, We write 2% for the set of all functions 9(2) 
(0<t< oo) for which the following condition is satisfied: if 
o(s)EG for a certain s, then 9()E€P for any t>s, By 
theorem 2,8, our ra will be proved if we can construct 
a functions (ft, x1, .. . Jt; 2 0, x,C E)that satisfies condi- 
tions 2,13 .A-2.15 ‘3B. 


We put 


Gls My eet Xn ee DS NM Xe (*;) Zr (*;). 
t< tj 


Obviously, the function 7 is #«“-measurable and non-negative 
and satisfies condition 2.13.A, Furthermore, 


Ms, 29 (f1 Xtys sees tn» Xtys + j= 
», Ps 2 {%H EG, *t,€ PD} = 
ti <t; 
= > [Pe x; ty, dy)P(ty y; tj, T)=0. 
ti <t,6 


Thus the function 7 also satisfies condition 2,13.B. 


2.15. The following theorem is also useful in investigating 
the properties of the trajectories of Markov processes, 


Theorem 2.9. Let X=(x;, ©, of, P, ,) be a Markov process 
in the phase space (£E, #) and let ZF "be an arbitrary subset 
of space 2... We suppose that there is associated with each 
pair s>0, x€E a set Q2°*%-@, having the following properties: 


2.15.4. For each w€2** there exists a function ¢(/)(€[0, 
)) of 2 such that z=—(w) and 9(4)=-x,(w) for all f€lIs, 


= (w)) + 
2,15.B. If AGo#® and ADQ”’® , then Ps, 2(A)=1. 


Then a Markov process X equivalent to X exists, all the 
trajectories of which belong to 


Proof, We put 
C* 7 (w) == max (C(w), 5), 
x ® (w) = x,(w) (CE [S, 90), o € 2). 


In accordance with sec, 2,1, the elements (xe; on, i> P P, x) 
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define a Markov random function xX” in the time interval 

Is, co) . Let X%? denote the Markov random function obtained 
from X*” by a restriction of the space of elementary events 
to the set 7 (by the remark at the end of sec, 2.11, the 
existence of such a function is guaranteed by condition 
2.15.B). The system X°* (s>0,x€E)is easily seen to be a 
Markov family. By using the construction described in sec, 
2.1, we can build up from this family a Markov process X in 
such a way (in view of condition 2.15.4) that all the traject- 
ories of X belong to Y, I+ may readily be seen that X and 


~ 


X are equivalent, 


2.16. We shall dwell in conclusion on some special problems 
connected with the subordination and equivalence of Markov 
processes, 


Let X=(x1, 6 oft, Pez) and X=(%; © off, Px) be stationary 
Markov processes, the process X being subordinate to X, 
Let 4, and %, be the corresponding operators satisfying con- 
ditions 2.5.4-2.5.C. Then 


B14 = 171,4 (AE dl). (2,32) 


For, 
17h x, ED} = (x, ET}. (2.53) 


The events A , for which relationship (2.32) holds, form an 
invariant system with respect to all set operations, By 
(2,33) and 2,5,B, this eee contains all the sets (x,€T}, 
It thus contains "gf * 


Theorem 2,10, Let c% be the class of equivalent Markov 
processes having the transition function P(s,x;t,T). The 
necessary and sufficient condition for class .#4to contain a 
stationary process is that 


P(s,x:t,T)=P(0, x; ¢—s,T) (OSs<t, x€E, PEF). (2.34) 


If condition (2.34) is fulrilled, the canonical*) process X 
belonging to o%f' is stationary. 


*)As is clear from the proof, any .f -canonical process is 


stationary, provided only that (Y satisfy condition 2.7.B. 
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Proof, The necessity of condition (2.34) follows from _ 
2,5.B and 2,5.C, Om the other hanéi, the canonical process X 
always satisfies condition 2.7.8. If (2.34) is satisfied, ¥ 
also satisfies condition 2,7.4 and, by theorem 2,3, is a 
stationary process, 


2.17. We shall now look on a stationary Markov process as a 
system (x, 6, of, P,, 4, satisfying conditions 2.8.A-2.8.H, We 
shall describe the stationary process X=(x, 0, of, P.., 6) as 
subordinate to the stationary process X= ey ee oH,, P,, 6,) if 
we can construct a mapping y of the space of elementary 
events @ of process X into the space of elementary events 
2 of process X such that: 


2.17.4. E(wy=f[y(@)]  (w€Q). 

2.17.3. X= aly @] GES, 0<t<F()). 

2.17.C.  y~'0h, S off. 

2.17.D of D1-'oM and P,(7-'A)=P, (A) for AG of"). 

Theorem 2,11, The process X=(x;, 0 o&;P,, 6,) is subordin- 
ate to process X=(x;,(, of; P,, 4,) when and only when there 
aan stationary Markov processes Re Gas 2 okt, Py, a) and 


X’ = (x, 6, off, Py.) Such that X’<~—>X, x’<»> x and y is sub- 
ordinate to xX’. 


Proof, The sufficiency of the condition is obvious, The 
necessity is proved with the aid of the construction used in 
proving theorem 2.4 (sec. 2.8). 


We shall say that a process X’=(x,,%, of, P,, §,) is canonical 
if X’<>X, where X=(x;, 1, of}, Py,2) 18 & canonical process 
in the sense of sec, 2,11, Two processes X = (x,,5, ,, Pz, 9) 
and X = (x, 5 r, ody, P.. 6.) will be called equivalent if their 
transition functions coincide, i.e. for all ¢>0, x€E, TECH 


Pa (x,€T} = Py (x ET}. 
Theorems 2,6 and 2.11 readily yield the following result. 
Theorem 2,12. -Every class of equivalent stationary Markov 
processes X = (x,, 6, oh, Po: 9,) contains one and only one canonical 


process X= (Xp 6, a At, P... 6.) a and consists of all stationary 
processes subordinate to a 


*)Relationship (2.32) follows obviously from these conditions. 


CHAPTER 3 
SUBPROCESSES 


1. Definition of Subprocess, The Connexion Between 
Subprocesses and Multiplicative Functionals 


3.1. We discussed in article 3 of Chapter 2 some construc- 
tions (transformations of the space of elementary events, 
widening of the basic s-algebras) whereby further Markov’ 
processes could be built up from a given process, All these 
Markov processes have the same transition function as the 
initial process, 


The present chapter is concerned with a type of transforma- 
tion in which the transfer function is altered, The essence 
of such transformations consists in a shortening of the life 
£ of the process, 


Let X= (xz, 5 oft, P,, 2) and X = (x;,, %, offi, P.,2) be two Markov 
processes, where 7 (w) <6(w) for any € @, x1;(w)=x,(w) for 
O<t<f(w)and fi = of! [2,] (2=(C>t}) . We shall speak in 
this case of X being got from xX by a shortening of the 
life, 


We shall speak of a Markov process X being a subprocess 
of Markov process x if xX can be obtained by shortening the 
life of some Markov process subordinate to X, 


We note that every trajectory of subprocess X is part of 
some trajectory of process *, Hence, if process xX is 
given in a topological measurable space and is continuous or 
continuous from the right*), the subprocess X possesses the 
same properties, 


Let 2&2 be the space of elementary events of process 
*)Process X is said to be contirucus if all its trajectories 
are continuous, i.e, if for every wé2 x;(#) is a continuous 


function of ¢ in the interval (0,f¢()) . Continuity of xX 
from the right is similarly defined. 
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X= (x4, 2, hi, Py, z) arid © the space of elementary events of 
process ¥—(x,, 2, oj, P,,,)- In view_of sec, 2,11, the neces- 
sary and sufficient condition for X_ to be a subprocess of 
X is that there exist a mapping y: +2 satisfying the 
following requirements (cf, 2,11.A-2.11.D). 


Z.1.A, 2 (uw) <CIz@)] (EQ). 

3.1.8. (=x, [1] @ES, 0<t <7). 

3.1.0. If A€oMji, then {7-1(A), C> tC of. 

3.1.D, of Dy (cM )and PB, , (7-1A)=P,y, , (A) for AG oft. 

We agree to identify every function ¢(w) in space 2 with a 
function : [7 (w)} in space 9, every subset A of space @ 
with the subset ;-!4 of space Q, and every system of subsets 
.F ={A]of_ space 2 with the system of subsets y-'cF = {y71A! 


of space &. In these circumstances conditions 3.1,A-3.1 D 
can be rewritten as 


Pe lgh cs 2S 

3.1.B°, M= xX (0<t <2). 

3,1.0°, IfA€oM}, then [A,% >t} € off. 

3,1.Di. of Dof* and P, ,(A)=P,, (A) for AC of. 

Our definition of subprocess is so far unnecessarily 
general, We narrow it down by requiring the following condi- 


tion to be satisfied in addition. 


3.1.58. For every «CE and any 0<sx 
Pio{S>tloh}=ai (a,c.2, Py 2) *) 
*)This relationship can be written in more detail as 
Pole thy ted} = 0% [7 (@)] (asc377 "Uy. Py). 


In the formula in the text, y-tw#® is equated with #*, 1-1; 
with , , measures Py. x oi y-L.a8 with measures Ps. on 
#° and finally, functions ef[1(0)]}(o¢% with functions 23 (w) 
(w € 2), 
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where a(w) is an yy3-measurable function*), 


The last condition may be pictured in the following way, 
When calculating the probability of the event %>1,a 
knowledge of the total aggregate of phenomena connected with 
process X and observed during time [s, oc) yields no more 
than a knowledge of the trajectories of the process during 
time Is. ¢], In other words, given a knowledge of the course 
of X during time [s,¢] an event {>t is independent of the 
remaining phenomena observed during time [s,co) , 


We shall put AC&; ifA€N", AG, and there exists BE dN? 
such that p, .(4\A4B)=P,.(B\AB)=0 for all x€E, 
Obviously, R; is a s-algebra in space 2,, It may easily be 
verified that condition 5.1.H is equivalent to the following: 

3.1.E°. For every x€E 

Py a (0 > tfoM |= ai (wv) (a, Cc, Q:, Ps, x): 
where ai(w) is an &-measurable function of w**), 


We note that it follows from M23KiDWN, Mads WN, 
A DRiDAN; and condition 3,1,E that, for every x€E , 


Py, 2 {F> t{o#'} =P, 2 {2 > t[oHi} =P. .{f> t] of*} — 


wd ~~ . oa ~~ ( wake 
=—Pp as >t] Ri} =P, {2 > tf oN} (2.0.2), Ps 2). 0 ) 

We notice further that, for any 0<s<¢ t x€E 9 
Peg (C>t]oM}=0 (ace, Q., P. 2): (3.2) 


*)Condition 3.1.E presupposes that function ajis independent 


of x. It would be sufficient to require that %(%,*«) be an 
Ni X # -measurable function of » and x, For in this case 
the function «j(+, x,()) is of;-measurable, does not depend on 
x and for every x differs from «j(#,~«) only on a set of 

P,,2 -mMeasure zero, 

**)We put Aco? if, for every measure » on the s~algebra # 
we can choose 4;, 4, "of AN such that 4;¢AGA, and Ps,p(A;) 
=Pe,p(Ao). We put gcW%.o if ao,ch% for every o>t, We 
leave it to the reader to show that the whole of our theory 
of subprocesses remains valid if the s-algebra ft is defined 
by si=WisoNot. The set of subprocesses is considerably 
widened with this definition and includes several important 
new examples, 
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For, since {{>1}E€o\*%, we have from 3,1.A and 1,6,H: 
P, a {2 > tlod} =P, 2 (C >t T> tfos’} = 
= Xo,Ps,2(6> t| o%*} (a,c, Q, P, a) 


3.2. Lemma 3,1. Let x = (Xp, t, Mi, P,, z) be @ subprocess of 
Markov process X=(x;,,C, of, Ps.) and let functions «a; be 
defined by condition 3,1.E (or 3.1.B°). Then for any BE of 


P,, 2 (B, O> t} = Ma, oY gts (3,3) 
for any o#-measurable function és 
Ma, 284% 5 = Mo, aba. (3.4) 
The transition function of subprocess X is given by 
P(s, x; t, T) = Me, 2@2{p(*2)- (3.5) 
Proof, By 1.6.F and (3.2), 


M,, aL? > i> M, zr {eP,, 2 (t> t|o#")} = M,, 


and expression (3.4) is proved, On putting ¢=7, in this, 
we get (3.3). We remark further that, by 3.1.B, 


(x,ET] = (x,EP, F> 4h. (3.6) 
Expression (3.5) follows from (3.3) and (3.6). 


Lemma 3,2, The necessary and sufficient condition for two 
subprocesses 


x= (x1, e A, Ps, =) & X= (x2, a Ai, P,, ) 


to be equivalent is that, for any 0<s<t, x€E, 


ae {T > tpod*} =P, {5 > tfoA" (a,c 2, Pp (3.7) 


8, an) 


Proof, The sufficiency of (3.7) follows directly from 
lemma 3,1, The proof of necessity reduces to showing that 


rw 


ota 2h (a,c, Q,, P., z)s (3 8) 


where a; and af are wW:-measurable functions corresponding to 
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subprocesses X and X by virtue of condition 3,1.B. We 
remark that, for any s<t,<ch<...<t=tts7T,,...,. 1,68 dy 
(3.3) and (3.6): 


Py, (SX; fy, r,, seen Ly, T,) = Peo (Xz, EY, aN x4 € Pn) = 
= Ps, a {¥e, ET ..-. 04,ET ye O> tf} = 
= M,, « [Zp, (*,.) baa ap (*, ) atl. 
Similarly, 
P(s, x,t Ty... A T= M, o[ur, (*,)--- Xr, (*, ) a]. 
But it follows by (2,15) from the equivalence of X and X 
that P(s, x; tT... fy Ty)=P(s, x; ty Ty... ty r,), and there- 
fore 


M,, 2 [(2% — at) kr, (%2,) +++ Xp (%2,)|= 0 (3.9) 


We write 2 for the class of all functions §(w)(w€2,), @ 
for the class of w-sets of the form 


{ET ... x1, ET} 
(41, 2,260.3 84, Coes SC tyet; Ty as BEG) 


and <¢# for the class of functions ¢(w) (w€&,) such that 
M, zx [(a3 — a8) é] —= 0. 


Obviously, C is a z-system, and &# an .%-system in space 
2. By (3.9), S@ contains the characteristic functions of 
all sets of @, and by lemma 1.2, # contains all functions 
measurable with respect to o(@)=WN{. In particular, # 
contains af—o., Thus M. 2(%@— a fP=0 so that relationship 
(3.8) is satisfied, 


3.8. The subprocess X= (xp, , offi, Ps, z) of a Markov process 
X=(x,t, Mf, P.,2) will be described as canonical if: 


3.3.4. The space of elementary events S of process X is 
connected with the space of elementary events & of process 


X as follows: 9—@yx/, where /=[0,+-co] and 1(w, A)=w 
(EQ, EN, 


3.3.B. 2 (w, s)—= min [%(w), A]. 


3.3.0, 4 N=x4(v) for 0<t<l(w, W). 
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3.3.D, oM; consists of all sets of the type AX(t, co], 
where A€ of}. . 


3.3.E, of is a c-algebra in space Q, generated by the sets 
(4,0 >th= [A, Co AKG CO} (AEC Kh’, t> Ss). 


Theorem 3,1. Every class of equivalent subprocesses of pro- 
cess X contains precisely one canonical subprocess X and coin- 
cides with the class of all Markov processes subordinate tox, 


Proof, a) We show first of all that equivalent canonical 
subprocesses coincide, Let X =(xp, 7, fi, B,,~) and X¥=(x, =, 

*, P,, 2) be two equivalent canonical subprocesses of process 
XxX, By 3.3.4-3.3.E, these processes have the same space of 
elementary events Q X/ and we have for them C—*% %,— x, 
Mi = Ah; and ff? = ch". 


Tt remains to show that, for any B€ofM*, 
P, -(B) = Py, » (B). (3,10) 
The sets 8 for which (3.10) holds form a )-system F in the 


space 9=Q—=—2\/. On the other hand, sets B=[A, [>1| - 
(¢, co] (ACM, t>s) form a z-system C, where ole. 
lemma 1,1, it is a question of proving "that F PC , i.e. 
that (3, 10) holds for sets B=[A, 6>¢] X(t, co] (AEM, ts). 


By (3.3), we have with any A€o#, t>s and x€E, 


24. 2>4X@ col} = 
ae: {> t]) X(t, cof = 


Poy (A, 5 >t} = My 174%?) 
P. {4 5>¢ 

where as and as are functions satisfying condition 3.1. for 
X and ‘x respectively, Since subprocesses X and X are 
equivalent, (3.8) is satisfied and therefore 


Pax (lds $ >) X G00} = Py, 0 (4s 5 > AX CF oO}. 


b) Obviously, if X is a subprocess of process X, all the 
Markov processes subordinate to X must also be subprocesses 
of X and equivalent to xX. 


It remains to show that every subprocess X = (Xp 5, Mt, Py) 
of process X is subordinate to some canonical subprocess X, 


Let us suppose that the space of elementary events © of 


subprocess is connected with the space of elementary 
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events 2 of process X by a mapping ~:@54Q. Then the 
expression 


1 () = (7 (), $()) (3,11) 
gives _the mapping of 2 in 9=-2x/:, We write e Xp os 
and oA@* for the elements defined by conditions 3,3.B-3.3.E. 
If we take into account the fact that elements ©, x,, of and 
oS’ are connected with elements %. x; cM, o#* by expressions 
3,.1.4-53.1.D, we can deduce the following relationships between 


ia Xe. Hi, A® and G Xt MM}, oh? ; 
Clq ()] =2 (0) 
HIT(= x (0) for + <2 Fey | (3,12) 
st a8 Us ° 
1 (Ai) & A$ 
1 | (of) & off 


ee 


We put for each A€oAf*: 


P,2(A)=P, fT (AD. (3.13) 


It may easily be shown that all the conditions 2,1.A-2.1.F 
are satisfied for (x,t, o#i, Ps2) . These elements thus define 
some Markov process X, Obviously, X is a canonical sub- 
process of process y, On combining (3,12)-(3.13) with con- 
ditions 2,11,A-2,11,D, we observe that the Markov process X 
is subordinate to process X, 


8.4, In view of condition 3.1.E”, for every subprocess of 
a Markov process X there is a corresponding system of func- 
tions a§() (0<s<t, w€@,) (functions %{ are measurable with 
respect to the c-algebra §i.). Here, every function a; can 
be varied as desired over any set belonging to &; and having 
P,.. -measure zero for any x€E. We shall speak of two 
systems {#{} and {9$] as being equivalent if a{=a! (a.c, 

Q, P;,.,) for any O0<s<t,x€E. By lemma 3.2, there is a 
one-to-one correspondence between classes of equivalent sub- 
processes of a Markov process X and classes of equivalent 
systems of functions {3} (if we consider only systems that 
correspond to some subprocess), Our main purpose is now to 
discover what sort of systems {2} correspond to subprocesses 
of a Markov process X, The first step im this direction is 
provided by the following lemma, 


Lemma 3,3. Let X=(x, % o&f%, P.»x) be a subprocess of the 
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Markov process X=(x, 6, off. Ps). The functions # defined 
by condition 3.1.E (or 3.1.E”’), possess the following proper- 
ties: 


3.4.4, For any s<ct<cu, «€E 


t 
oh, = OYOte, (A.C. Qa P, 2). 


3.4.B, For any s<t, x€E 


0<a<i (a.c. 2, Pe cc): 
3.4.C. For any s<t, x€E 


Ace ot == oy (a,c, 2;, P, ;). 
n 


Proof. Property 3.4.B follows from 1,6.B,, and property 
3.4.C from 1.6.D, It remains to prove 3.4.A. 


We shall show that, if AEN}, BEN then 


wu 9 
P, (AB, 6 > u} = My. oe [Xaneiat]: (3,14) 
We note first of all that 
(B, T> ua} € A. (3.15) 


where of, is the s-algebra in space @, generated by sets 
(x,€T. C>u} Eft, u,.TE#) . For the class of all subsets 
B of set &, that satisfy (3.15) is a c-algebra in 2. By 
(3.6), this c-algebra contains all sets {x,€I, C>z} (vE[t, ul, 
T'ég@) . It thus contains Wi. 


Since {A,C>t}€o#, (B,C> u}EoVn , we have on applying (2.9) 
to the process ¥: 


B, . (AB, €> al} =P, (A. C>t, BO >a} = 


= : (3.16) 
= M, {Xaz> P,3, (B, > ut). 
Moreover, by (3.3), 
P,y(B. C> a) —=M, , (zy2t). (3.17) 
We have from (3.4), (3.16) and (3.17), 
P, (AB, t> 2) = Me, [2% 4Ms, © (X2%,)]- (3.18) 


On the other hand, we find on applying (2.10) to the 
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process X: 
M, [XateLa%,| = M, [x40eM, z, (45%,)]- (3.19) 
Qn comparing (3,19) and (3,18), we get (3,14), 
Let # be the class of events DcQ, , for which 
P,2(D. C> wo) =M, , [xpazat]. (3,20) 


and let @ be the class of all events AB, where ACW, 
BEN, » We have already shown that ¥>@. It is clear 
that @ is a z-system and ¥ a j-system in @,, and by lemma 
1.1, ¥ 2s(@). But obviously, o(@)—WN%, Hence ¥ DWN, 
Since (3,20) holds for any D€d% and since function aa’ is 
clearly o";,-measurable, we have 


Poll >ufWa} ain (a,c, 2, Py.) (3,21) 
On the other hand, by (3.1), 
Pag {e> uf Ne} = on, (a,c, 2, Ps 2): (3,22) 


Expressions (3,21) and (3,22) yield 3.4.4, and the lemma 
is proved, 


‘ 3.5. We now take the equivalent systems {#} connected with 
@ given subprocess xX of process X by condition 3,1,E”’ and 
attempt to distinguish the one for which conditions 3,4,A- 
3.4.C are satisfied in a stronger form (not for almost all, 
but for all w), 


We shall speak of the system of functions %(w(0<s<t, 
w€2,) as defining a multiplicative functional « of the 
Markov process X=(x;,,C, of, Psc) if the as(w) satisfy the 
following conditions, 


3.5.4. at(w) is &R-measurable, 

De5eB. af (wat (w)=ak(w) (stu, wEQ@,). 
DoH, Oa) <1 (s<t, wEQ)). 
25D. iim of, @)—= a4 (0) (s<t, w€ Q). 


We shall say that the subprocess X of Markov process xX 
corresponds to the multiplicative functional « if, for any 


62 THEORY OF MARKOV PROCESSES 


S<t, xEF 
Psa (C>theM} =ai(w) — (A.6. Dy Po, 2)- (3.23) 


Paces AZ. The sufficient condition for the subprocess 
== (x; C, .oMj, Psa) Of Markov process X to correspond to 
te multiplicative functional is that either of the follow-— 

ing two conditions be satisfied: 


3.5.0. P(s,x;s, E)=P,-@>s)=1 for any s>0, x€E. 


3.5.8. The functions a$ defined by condition 3,1.E° are 
subject to the convergence 


Sn = at (a,c, Qe, P, 2) 


lim O, 


8,48 
for any s<t, x€E. 


Proof, a) If P,,{0>s}=1, then by 1,6.B,, 
af = Pe2{i> sot’ = 1 (a.c, 2. P, a) (3 24) 


Let s,{s. By 3.4.4, 3.4.0 and (3.24), we have for, almost 
all w€2, (in the sense of Pas a, == ay oat as > Os 
and g’-—1, Condition 3.5.8 follows from this, 


b) We now show that, as a consequence of 3.5.8, the sub- 
process corresponds to some multiplicative functional a, We 
shall start from any system of functions «af defined by con- 
dition 3.,1,E° and construct an equivalent system of functions 
yielding the multiplicative functional «, 


The set 
= {a < 0} U {af > 1} 


belongs to &#; and, by 3.4.B, P., (At) = 0 . We replace the 
values of functions «f(w) on the set A; by zero, We get the 
equivalent system of functions which satisfies condition 
3.5.C. (Naturally, it also satisfies condition 3.1,B’ and 
conditions 5.4,.4-3.4.C) Our initial functions will be denoted 
by % again, 


We put 
Qe tu {HF oath (s<t<uy). 
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Obviously, Qs, £ wel, By (2, a) for any g<s we have 
Pan (Qs, t, u) = hae Ts (Qs, t; u) Pan (dw). 


In view of 3.4.A, Psy (Qs ¢, ar for any y, and therefore 
Po a (Qs, t,u) = 0 for any xX. 


Let %(w) denote the exact upper bound of rational numbers 
such that €Q,,:,. for certain rational u>t>s. Clearly, 
n(w)< S(@) and 


{7 > 9} = U Qs, t,u 


g<s8s<t<u 
(summation over rational s,t. 4), Hence it follows that, 
for any <€E, 
Po,x {1 > 9} = 0. (3.25) 


Condition 3,5.B is evidently satisfied for any rationalu>/>s 
of (4.0). Hence for any w€Q, and any rational t€ (7, 0),28(w) 

is a non-decreasing function of s (if s rms through rational 
mumbers of the interval ((») ‘)), Thus the limit exists 
for any real s>7(w): 


a (w) eee (w) (p rational ). (3,26) 
It is clear that 
gt asiag’ (q(w) <s<q’ <q<C(), g and 4’ rational ). (3.27) 


Hence a!+° is a non-increasing function of ¢ (for rational 
values of ¢€(s,0)). We put 


aio limayt? rational. 1<s<t<0). (3.28) 
If s<t<u are any real numbers from [»n, (), we obtain by 
putting first g’ | ¢ then 9] in (3,27): 


8+0 — 78+0,%+0 
On +0 = i 044,40" 


Let s<t<€(wj). We put 


ag+8(w), if qw)<s 


cas | 0, if yw)>-s. 
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These functions may easily be seen to satisfy conditions 
3.5.B-3.5.D, Furthermore, 


{at = a3} = C,UC,, 


where 


C,={n> s\, C= {7 <s, at # afl. 


«= 


By (3.25), for any x€E, 
P, 2(C) = 0. 


By (3.28) and (3.26), 
as {U2 en a) T { lim ap af } 


qg ° 


(p-<q ave rational numbers), By 3.5.8 and 3.4,C, we now 
conclude that P, ,(C.)=0 and therefore 


P, , (a # af] = 0, (3.29) 


Function a! is clearly o-measurable, In view of (3.29) 
it is &-measurable and satisfies relationship (3.23), 


The theorem is thus proved, 


3.6. Theorem 43,3, For every multiplicative functional 2 
of the Markov process X=(x, © of;, P,,.) there is a corres- 
ponding subprocess X of X, 


Proof, It follows from theorem 3,1 that, if one subprocess 
corresponds to the functional a, there is a canvunical subpro- 
cess corresponding to a, We shall therefore construct sub- 
process X in the enon cal form, We define ©,7, x,,o#; and 
oh? by means of conditions 5,3.A-3.3.H, It remains to assign 
the measures P, .. 


We fix w€2 and s>0 and consider the function 
0 for 9<ft<s, 


Fegé)=4 |1—at(w) for s<t<f(o), 
] for t> 4 (w). 
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In wiew of 3.5,.B-3.5.D this function satisfies conditions 
1,11,A4-1,11,0, and in accordance with sec, 1,11, there 
exists on the segment [0. co] the probability measure 28 
(TE #°) *) such that an t) = F(t) and therefore a co) = for 
wEQ,. - 


Let A€o#*®, Then ACoM* x #*, and in accordance with 
lemma 1,4, the set 


Ay = {X:(w, a) € A} 
belongs to #°S # for every w€Q2 . We put 
a4 (w) = a4, (w). (3.30) 
Obviously, for A=[C,C>4] X(t, co] (CE of’, t>s) 
a4 (0) =yo(w) aw)  (wEQ,). (3.32) 


Hence we easily deduce by conventional use of lemma 1,2 
that the function a(w) is of*-measurable for any A€oh*, We 


put 
P., 2 (A) = My, 2. (3,32) 


Clearly, for every A€o#*, 


Pax (AX N= Py, 2 (A). 
It follows from (3,31) that 


P, 2 (C,C>e} =P, {IC C> AX ool] = 


=M,,.xop (CEM, t>s), 933) 


The system of elements (Xp. 0. of, Pe, az) evidently satisfies 
conditions 3.1,A-3.1.D. It follows from expression (3,33) 
that 


Peo {t >to} =f (2.0. Qy Pe, 2) 


so that condition 3.1,.E° and relationship (3,23) are satisfied, 


We also want to show that elements (x,t. oM. Py.) satisfy 
conditions 2,1,A4-2.1.F, It is obvious that 2,1,A, 2.1.3 and 


*) gs denotes the c-algebra in the space [s,+ oo] generated by 


all intervals. 
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2,1,C hold, It follows from (3,33) that P(s. x; 4,T)=M, ,a%7,(x,). 
Since the function Zp(+;)a; is \’s-measurable, it readily 
follows from lemma 2,2 that P(s, x; 4,.T) is a #@-measurable 
function of x, so that condition 2,1,D is fulfilled. It is 
obvious that 2,1,E holds, It remains to prove condition 

2 pets 


We can more conveniently prove this last condition in the 
form 2,1.F’*. In accordance with 3.3.D, every set A of off§ 
has the form A=C>(t, co] (CE eM) . By (3.33), 


Py n(A, x,€T) =P, , (IC, x, ET) XK (a, co}} = (3.34) 
—s Mg, Zz (Xo%r (X,) an]. 
We have on the basis of 3.5.B and 1.6.H, 


Me, x [Y¥o%p Hu) %] = Ms, 2» [986% p (¥u) OF] = 
= Mg, v (7coSMs, x [at Xp (4) j oAf®}}. 


(3.35) 


t 


on applying theorem 2,1 to the process X: 
Mg, x [a0 up (x,,)| Al:| — M;, ”; [ety (x,,)] = 


The function a’y,(*,)is V'-measurable, We therefore have, 


= P(t, x,; u, T) (a,c, &,, Py, 2). (3.36) 
We have from (3.34), (3.35) and (3.36): 
Peo (Ar yET)=My, 2 (x(08P(t, xy us D)}. (3.37) 


It may easily be seen from (3,33) that (3.4) holds, 
Relationship (3.37) can therefore be rewritten as 


P,. 2 (A, Xy€ P= Me, ae (Xores P (t,x; 4, T)} = 
=M, 2[xaP , %,; uw, TY), 


and condition 2,1,F° is satisfied, 
The theorem is now proved, 


3.7. We can connect with every multiplicative functional a 
of a process X a system of random variables &, which are 
defined as follows: 


inf (¢:a¢(w)—=0} (wE€Q,), 


2 (3.38) 
s (w€Q@,) 


Es (w) = 
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(if of >0 for all t€ [s, 0), we put §&(w)=¢), In view of 
3.5.D, the lower bound is attained in (3,38), so that af —0, 
Some obvious properties of the functions { may be noted, 
3.7AL SShs(0) <4 (w). 
3.7.B. (8 >t} ER (OSs<P). 
3.7.C. s> 4} S f= 8) O<s cd). 


Lemna, . if X is any subprocess of process X corres- 
ponding to the multiplicative functional «a, we have for all 


s>0, x€E 3 
Ps a(t <i j=l. (3.39) 
Proof. By 3.7.B, 1.6.F, (3.2), (3.23) and (3,38), 


Pe ait >t>b,fot’} = =Xi>e,? Poa © ~ > tlo#s} = 0 (3 40) 
(a,c, 2, P, ade : 


Consequently 
PB 2 (t> i> E.| = 0, (3 41) 


Let A be a denumerable set everywhere dense in the interval 
[S, CO). Then 


(7 > §,) = Uté> t >t}. (3.42) 


ewe (3.39) follows from a comparison of (3.41) and 
3 42 @ 


We shall consider in detail the important particular case 
when the functions @ defining the multiplicative functional 
a take only two values: zero and unity, In this case the 
functional a is expressed in terms of functions § by 


(= 7% 5 ()- (3.43) 


If {§,} is any system of functions satisfying conditions 
3.7.A-3.7.C, the functions %—X«,>t will satisfy require- 
ments 3.5 res 5.D. Formulae (3, 38) and (3,43) thus define a 
one-to-one correspondence between multiplicative functionals 
with values zero and unity and all the possible systems of 
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functions § subject to conditions 53.7.A-3.7.0. 


A comparison of (3,5) and (3,43) shows that the transition 
function of the subprocess is expressed in terms of § by 


P(s, x; 4, D=P, 2{x, ET. & > 2}. (3.44) 


Lemma 3,4 can be strengthened as follows in the case in 
question, 


Lemma 3,5. if X is any subprocess of process xX corres- 
ponding to the system {5}, we have for all s<JU,*x€E 


Be a lig>>s, THE} = (3.45) 
and for all x€E 
Po, a (6%) =0. (3.46) 


Proof, For any ‘>s, 
Be, (CSE <8 No] =, ce Be, 2 (0 < tI of") = 


= Mee, iy I= 9 .0.2,P,, 2) 


(3.47) 


If A is a denumerable, everywhere dense subset of the 
interval [s, oo) , we have 


[&e>s C<Bl =U <1 <h}. (3.48) 


ZEA 
It follows from (3.47) and (3.48) that 
P,, w {bs > s.6 <&,} =0, 
and we get (3,45) on taking into account (3.39). 


If s=0, the set [&<s,C<§,}=—{%=—0,0<0! is empty, and 
equation (3,46) therefore holds, 


2, Subprocesses Corresponding to Admissible Subsets. 


The Formation of Parts of a Process 


3.8. Let I be a subset of the phase space (£, #). We put *) 


* Writing x;(wyer indicates that the value of x;(«) is either 


not defined or else belongs to EN , 
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és (I) = be (To) = ink (£248, x,(w) EF} WE) 
cae 1. (5.49) 


Let ¢<C(w). The segment of trajectory +, (¢<4<Ais con- 
tained in fT if t<£ (r), and is not contained in [ if 
t>§&(W) . It is therefore natural to speak of £& (I) as the 
instant of first departure after s of the trajectory from 
set [, The functionst,—§& (always satisfy conditions 
3.7.4 and 3.7.C, If they also satisfy condition 3.7.B, we 
shall describe the set [ as admissible, Some subprocess of 
process X corresponds to each admissible set, 


The question as to whether a given set is admissible for a 
process X has to be considered on its merits in each case, 


Lemma 3,6, The sufficient conditions for a set [ to be 
admissible for a process X are thats: 


3.8.4. VW= ne {x,€T} € Ri. 
3.8.8. (8 (I) > ¢} E ol and 


P(t, x; t, EY= Py 2 (80) >t} =1 for all ¢>0, xe€P. 


The transfer function of the subprocess corresponding to 
set [’ is given by 


P(s, x; t, G)=Ps, 2 (Vir % EG}. (3.50) 
Proof, We have 
{Er > ft} = (Vy & (> ¢}. 
Hence by theorem 2,1, 


Pex (0) > t}—= f Py», ((L) >t} Py, g (do) = Ps, 2 (I). 


8 
¥ 4 


Thus 
Po (WENT @) > 4} = (3.52) 


and fa > t}€ Ri. Formla (3,50) follows from (3.44) and 
3451} 


We note that, if process X is continuous, ~: (7,.) (w) belongs 
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to the boundary of I'*) for every w satisfying the inequality 
s<t,(,o)<C(w). If G is an open set, the boundary of G does 
not intersect with G and therefore 


fw: a (G, w) < C(w)} = { ,M > Xe g (Fo) € ENG}. (3.52) 


Let 7 be an arbitrary w-function, It is natural to speak 
of §& ()=£,~)(I, ©) as the first instant of departure after y 
from set [., The reader will easily prove that 


eT) >y>s} S (3,7) =, 1}. (35555) 


If Gor ,we have s<é,(G)<é (Mr, and it is clear from 


(3.53) that 
EC) =k@l) Wwe g). (3.54) 


The question as to whether set I. is admissible is closely 
bound up with the question of the measurability of events 
vs (T) = {segment of trajectory x,(s<u<?) is contained in [}, 


If process X is given in a topological measurable space, 
it is important for us to investigate, in addition to events 
Wir) , the events: iv (1) = {closure of segment of trajectory 
xy(S< uct) is contained in [}, 


A general investigation of tne conditions of measuranility 
of events Wr) and W:(T) requires the use of the theory of 
continuation of capacities, The subject is therefore treated 
separately in a special supplement, 


3.9. When we construct a subprocess of a Markov process to 
correspond to an admissible set E£, the phase space remains 
unchanged, This naturally suggests a modification to the 
construction so that the phase space is shortened to the set 
BE. 


Let X= (x; 6. oft, Ps.x) be a Markov process in the measurable 
space (F, @)and let Ee # and g= ol: We shall describe 
the Markov process X= (Kp C, Ak, a) in the phase space 
(E, #) as part of process xX if there exists a mapping 
": ©-+@2(Qand .Q are the spaces of elementary events for Xx 
and X respectively) such that: 


*)The boundary of a set ris defined as the intersection of 


the closure of lf with thie closure of ENI, 
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3+DAe Tote 1@)] WE 9). 
3.9.B. x (=x [7(o)] (ES, 0<t<l(a)). 
3.9.0, If A€oM: , then (7A, C>12} € off. 
3.9.D, of? D1-*(of*) and P, ,(7-'A)=P, 2 (A) for AC oft, x€E. 
The transition function of process X is given by 
P(s, x; t, T)= Ps {xET. te >t} (KEE 0<s<. (3.55) 


Theorem 3,4, A part of process xX can be generated on 
every non-accessible set satisfying condition 3,8.B, 


Proof, Since 
Wi = (x,€ E} CRY, 


the set E satisfies condition 3,8.A, It is admissible by 
lemma 3.6, We now consider the subprocess Y= (x,, [ off, P,. nl 
of process X corresponding to the admissible set F, We put 
E,=6,(E), f= t(B, 2= =e, Fo)=o (OEY). 
Since FE is non-accessible, we have 
(8 > s} & {E=&}. 

Thus 

GD (C=t t>s} = (78, >5}. (3,56) 

Let A€o#® and A >. We now have, in view of (3.56), 
ABl{E=&, > s} ={% >s}\ fb # be & > 5} 
and, by 3.8.B and (3.45), Ps, P,, (A) =1 for every x€E, Hence 
the ’ probability measures P, ,(x€E) defined in the c-algebra 
induce the probability measures P, ,2 in the o-algebra 
=—7-!om# (in space Q) (cf, the proof of theorem 2,5). 

fe put 


~- 


ot = oA, Co) =C fy (o)). x, (0) = x, [7 @)]. 


The elements (x, ©, of, P,,) obviously form a Markov process 
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in space (£,¢). This process is readily seen to be a part 
of process X, 


The theorem is thus proved, 


3.10. If E is any admissible set for process X and if X¥ 
is any subprocess corresponding to _&, there exists 3 by virtue 
of (3.44) and lemma 2,4 a process X equivalent to X for 
which the set F is inaccessible (lemma 2 4 is easily rewrit- 
ten for cut-off processes), Suppose that F satisfies condi- 
tion 3.8.B, Then by theorem 3.4 we can form a part X of 
process X on F, Process X will not in general be a part 
of process x, however, since the passage from X to the 
equivalent X will generally involve a widening of the space 
of elementary events (and the set of trajectories) and a 
narrowing of the basic c-algebras ofj, of’, In order to 
generate a part of process X on the set E , certain restric- 
tions need to be imposed on X, viz we must require that its 
set of elementary events be not too narrow, or its basic s- 
algebras too wide, 


Theorem 3,5. Let the Markov process X=(x, 6, of, P,,,) in 
the measurable space E€ # and the set(E. #)satisfy condi- 
tions 3,.8,A and 3,8,B and the following conditions: 


3.10.4. ofs—WN2. 


3.10.B, If +,(o)€F , there exists an w’ such that x,(w’)€F 
for all w€[0, s] and £0’) =C(w), x, (w’)=x,(0) for all w€[s, €(w). 


A part of process X can now be generated on set E ° 


Proof, The set E is admissible by reason of lemma 3,6, 
Let X=(x,, 6, off, P P,, z) be the canonical. subprocess of X 
corresponding to —, As in the proof of theorem 3.4, it 
amounts to showing that the set O= (T=—£} satisfies the con- 
dition: If A€ of and Aze, then p, .o(A)=1 for all x€E, 
In our present case, G— a — min Q, 0), MS hx FP= 
NY xX BF. 


We put w€A, if (w, NEA, By lemma.l.4, A,CN¥., The 
inclusion 4>9 is equivalent to the system of inclusions 


A, 2 {minQ, Q=§} AED. (3.56%) 


We put C€# if C possesses the following property: as 
soon as wEC, C(w)=C(w’) and x, (w) = x, (w’) for all wE[s, C()), 
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then o’€C. Evidently ¥ contains all sets {x,¢€T(t>s, 
TC #) and is a c-algebra, Therefore ¥ D \, 


Let }>S$ and let §& (w)=§,(E, o)=minfA, C(w)). By virtue of 
condition 3.10,B, an w’ exists such that ((w’)=£(w), ¥(w’) 
=*,(%) for all u€[s,o(w)) and x,(w)€E for all u€[0, s}. 
Evidently §(w’)=&(E, w’)=& (EF, w=min{i, C(w)I. By (3.56%) , EA, 
“ since A,€o/* we have w€A,. We have shom that, for all 

7S 


A, > {§, == min fA, C(@)}}. (3.57) 
It follows from the system of inclusions (3,57) that 


Therefore 
P, x (A)>P. 2 (C= fy > s} =P. 2 Se > $} oF 
—Ps oth # ho Fs > s} =! 


[by 3.8.B and (3.45) ]. 


3. Subprocesses Corresponding to Admissible 
Systems of Subsets 


3.11. Let F bea system of subsets of the measurable space 
(E, PF) ° We put 


Ea (oF ) = 5, (F | w) == sup ET, @) (w € Q), 
CEs (3 58) 
5 (oF ) = bo (oF ) = sup § (1). : 
Tee 


Let Gz denote the sum of all sets of system ¥, It is 
easily seen that 


bo (F) < & (Ge). (3.59) 


We shall call §,(¥%) the first instant of departure after s 
of a trajectory from the system of sets F, 


The functions §,(¥, w) always satisfy conditions 3.7.4 and 
3.7.0. If they also satisfy condition 3.7.B, we shall say 
that system ¥% is admissible, Im accordance with sec, 3.7, 
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for every admissible system F there is a corresponding class 
of equivalent subprocesses xX of process xX for which 


B, nS = be (F)} = I. (3.60) 


We shall describe system ¢ as subordinate to system ¥ ‘if 
there exists for every setTC€ ¥ a setI”C€ ¥ containing it, 
It is evident from this that &,(¥)<&(¥’)for any s>0. It 
may be remarked that all subsystems of ¥ are subordinate to 
F. We shall refer to ¥ and Fas equivalent systems if ¥ 
is subordinate to ¥’and subordinate to ¥, In this case, 
Co (F)=',(¥%’) for any s, so that equivalent systems define 
the same class of subprocesses of X, 


On replacing s in §,.(%, w) by any w-function y(w), we get 
a quantity §,(#%) which it is natural to call the first 
instant of departure after 7 of the trajectory from system 


&F. We have from (3,53) and (3,58): 
EF) >a=—Un=s (F)>s}e 


eUin=s. §F)=',(F)} = (E(F) = 8, (F )}. (3.61) 


3.12. We shall describe the system of subsets ¥ of a topo- 
logical measurable space (£, @., @) as normal if an equivalent 


system [,,...T,, ..., can be found satisfying the following 
conditions: 

Debeete, Tysrswa Ty cos: -C@, 

3,12,B, Iy,...,7,, ... are closed, 


3.12.C, For any n there exists G6,€ @ such that l,<G, 
SPy+t. 


We observe that 
nas: 2 e 
eee (5.62) 


Some important examples of normal systems will now be dis- 
cussed, 


3,12,1, Let (E. p, #) be a metric measurable Space and let 
F be the class of all bounded sets of this space, We 
choose any point @ in E and put - PT, =({x:p(x, a)<n} . The 
sequence {I',} is obviously equivalent to system and satis- 
fies requirement 3,12.B, We see on putting Ga=dxip(x, a) 


SUBPROCESSES 75 


<nts 5} that it also satisfies requirement 3,12.C, We can 


satisfy requirement 3,12,A simply by supposing that # >@or 
that p(x, a) is a #-measurable function of x, 


3.12.2, Let G be an open set in the metric measurable 
space (E, p, @) and let ¥ be the class of all sets Y for 
which p(T, E\G)>0. The sequence P= x: o(% ENG)}>— 


is equivalent to ¥ and satisfies conditions 3,12,B and 
3 412.0(f x : p(x, E\G)> 5-5, }) can be taken as G,), If #2@, 


the continuity of the function p(x, E\G)(cf. sec, 1,8) 
implies that condition 3,.12,A holds, 


5.12.5. let the topological measurable space (F, @, g) 
have the following properties: 


3.12.3.4, For every point x a measurable bicompact set Q 
and a measurable open set.U can be found such that x€UCQ,. 


3,12,.3.B. A Sequence of eee ents bicompact subsets E,, 
exists such that F= UJ E,* 


n=l 


We shall write ¥ for the system of all measurable bicon- 
pact subsets of the space (E, @. #). 


We note that, for any setTC ¥#n@, there exist sets UE CH 
8B and 6 Ff) # such that TeUcQ. For, by 3.12.3.A4, 
for every x€I there exist U(x)€ Cf) #and ODE F/) @ such 
that xEU (x4)/SQ(x) . Sets a. form a covering of [, anda 
finite covering U (x) . 2 U (Xm) can be chosen from "this, 
Sets 7; Yu and Q= -U Q(x;,) obviously have the propert- 


ies that a require, 


We start from the system £, defined in condition 3,12,3.B 
and construct sequences [,, and G, by the following recurr- 
ence method, We put T,=—E&£,. Assuming set [,, to be already 
constructed, we choose sets G,€ CM and Q,€% # such that 
Mn,66,5Q, , and put IPy41=Q,UEn4i1 . Sequences {[,,} and {G,} 
obviously satisfy conditions 3,12,A-3,12.C, Sets G, more- 
over form a covering for any TCE, and since a finite 


*) Property 3,12,.3.4 is a strengthened type of local bicom- 


pactness, whilst 3.12.3.B is a strengthened type of s-bicom- 
pactness (cf, sec, 1 1). 
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covering can be chosen from this, and G,c...€G,¢..., we 
have ToeG,Sl,41 for any n, It is clear from this that ({I,} 
is equivalent to #., 


3.12.4, Let G be a subset of the topological measurable 
space (E, @, g) satisfying the following conditions: 


3.12.4,.A. For every «€G a measurable open set UGG and a 
measurable bicompact set QeG can be found such thatx€Gcag, 


3.12.4.B. G=[JE, , where the £, are measuratle bicon- 
pact sets, n+1 


On repeating the arguments of example 3,12,3, the system 
of all measurable bicompact subsets of set G will be seen to 
be normal, 


Lemma 3,7. let # be a normal system of subsets of a topo- 
logical measurable space (£,@,9). NowGzs€#f@. The con- 
verse holds if space (EF. @, #)is subject to requirement 1.9.B: 
every set G€ #/)\@ can be written in the form G—G,, where 

is a normal systen, 


Proof, The first assertion is obvious, To prove the sec- 
ond, we consider a measurable continuous function f such that 


G={x:f(x)> 0}, 
and put 


K={*:f()> -} (n=1, 2,...). 


It is easily seen that F yan. is a normal system that satis- 
fies the condition Gz=—G (we can take , 2 as 
Gn). [*:F@)>seo4} 


Lemma 3,8. If the system F consists of compact sets, it 
is subordinate to any normal system satisfying the condi- 
tion Gy oG, " 


Proof, let If, be a sequence equivalent to ¥ and subject 
to conditions 3,12,A-3,12,.C, and let G, be the sets described 
in condition 3,12.C, The system {G,} is a denumerable cover- 
ing for any set TE. Hence a number 7 can be found such 
that ToG,cTl,u1. System # is thus subordinate to the 
sequence {I',} and therefore also subordinate to ¥, 


SUBPROCESSES 17 


Corollary. LetGe@fC. All the normal systems ¥ con- 
sisting of compact sets and satisfying the condition G;=—G 
are equivalent to each other, 


3.138. Let X. be a Markov process in the topological measur- 
able space (F, @, @), where the space satisfies 1,9,B, and 
letGE€ CN#. In accordance with lemma 3,7, a normal system 
& can be constructed for which G,—G. We agree to write 


t,(G) = 8, (¥) (3,63) 


and to call t,(G) the first instant after s of departure of 
the trajectory from the interior of the open set G, The 
value of t™,(G) is dependent in general on the choice of system 
oF as well as on G. As we have just seen, however, all sys- 
tems # consisting of compact sets lead to the same value of 
t,(G). We shall see later (cf. lemma 3,10) that the proviso. 
regarding compactness..ceases. to have any importance when the_ 
process X is continuous, 


We shall write «(G) instead of +(G) and +,(G) instead of 
E(F)(G=Gs) - 


Lemna . Let X be a Markov process continuous from the 
right in the topological measurable space (F, @, ¢). If 
U, GE Cf\# and the closure of U is compact and contained in 
G, we have 


T_ (7) (G) = t, (G). (3.64) 


Proof, It follows from (3.61) that, for any function 7(w) 
(w€ 2.) 3 
(ts(G) > n>s} & {t, (G) =, (G)}. (3.65) 


Let = {T,,} be a sequence satisfying conditions 3.12,A- 
3.12,C and such that Gz=G. Let G, be the sets defined 
by condition 3,.12.C, The same method as used in proving 
lemma 3,8 shows that, for a certain m, the set G,, contains 
the closure of U. Hence it follows that UcI, for all 
n>. We note that x,€U for s<t<x,(U). If ,(mEG, 
we have *,€G, for some n>n, and, in view of the fact that 
the process is continuous from the right, there exists «>0 
such that x,€G, ©ln4: for alls<t<1,(U)te. Consequently 


{%2_ (7) € G} & (1, (G) > 2, (U)}. (3.66) 
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On putting 7—7,(U) in (3.65) and comparing with (3.66), 
we get 


{., (ry E G\ St (G) =*,, (0) (G)}. (3.67) 


Expression (3,64) follows from (3.67) and the obvious 
inclusion 


(90 € GS [6 (G)=%—(U), ta) (G) = 15 (G)}. 


3.14, Lemma 3,10. A normal system of subsets ¥ of a topo- 
logical measurable space (F, @, @) is admissible for every 
Markov process X continuous from the right, If xX is con- 
tinuous, we have 


E(F)='t(Gr) (s>0). (3.68) 


Proof. a) We consider the sequence {I}, equivalent to # 
and satisfying conditions 3,12,A-3.12.C, We have from (3.62): 


{E(¥) >t} = Ut. (T,,) > t}. (3.69) 
In accordance with sec, 3,8, we put 
G(T) = ‘a {x, ET}. 
wE[s ft] 


If wEW¥i(,), and consequently x,(w)E€T, for «€[s, ¢], the 
fact that X is continuous from the right means that a é>0 
exists such that x,(w)€G, for 4€[s, ¢+%), Hence §& (41,0) >, 
Therefore Wi(T,) & (&(Tn41) >t}. It follows from the inclusion 


7(n)& {& (Tae) > t} FTPs 
and formula (3.69) that 


(3.70) 


“If A is any denumerable cverywhere dense subset of the seg- 
ment [s, ¢] containing the point ¢, we have 


WEP) =P) (xu€ Pa} € vt. Gi) 


ueaA 


It follows from (3.70) and (3.71) that {&(#%)>2}E N72 . 
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The first assertion of the lemma is thus proved, 


b) In view of (3.59), we can prove (3,68) simply by show- 
ing that 


E. (F) > §, (G). (3.72) 


This inequality is trivial in the case when §,(¥%,)> 
Cw). If (Fw) <Cw), it follows from the fact that now 
X(7)EG . We show that 


(bs (F ) <O}S {xe, (sr) EG). 
On putting t=s in (3.70), we get 
(s(F) >s}= UJ Y0,) = U (x. € Ty! = (x, EG). 
n=l n=l 


It is clear from this that (%(¥)=s}S{x, .5,€G), It 
remains to consider the case when SEF. w) <C(w). By 
(3.62), starting from a certain n we have 5 <£,(P,)<C(w). 
It now follows, as shown in sec, 3,8, that ~¢,(r,) belongs to 
the boundary of [,, and, in view of 3,12.C, #5 SOx for 
m<n., On letting n-oco and taking into account the fact 


that ate (Tn) *8 ) » We conclude that x, ix) ES, for any 
at 


m, So that x, 2€G. 
8 


Corollary. Let xX be a continuous Markov process in the 
topological measurable space (F, @, #), where the space 
satisfies condition 1.9.B. Every set GE@CN #is now 
admissible for process X, 


For, in view of lemma 3.7, there exists a normal system ¥ 
such that G=Gs;. By lemma 3,10, this system is admissible 
and & (#%)=& (Gz). Therefore 


{Ss (G) > t} = [8 (F) > t} ER, 
which is what we wished to prove. 


3.15. Let X=(x;, 6, Mi, P.,2) be a Markov process in the 
measurable space (£, #), a system of subsets of the space 
E ,G=G y,and = g[G]. We shall describe the Markov pro- 
cess X=(x, 0, of}, P.,2) of space (6, #) as the part of process 
X corresponding to the system ¥# if there exists a mapping 
1:832 (6 and Q@ are spaces of elementary events for 
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X and X) such that: 
3.15.4. C@)=tF. 1] WER). 
3.15.3. x,(o)=%[7@)] @ER, 0<t¢ <fo)). 
3.15.0, If A€oMi, then {77'4, F>t} € of. 
3,15.D. of’ D1‘ (0A’) and B,,.(77'A) =P.,2(A) for ACof’, x EG. 


When §(%)=—t(G,z), this definition becomes the definition 
of sec, 3.9 for a part of process X on the set G;z. 


The following theorem is based on lemma 3,10 and is got by 
repeating almost word for word the proof of theorem 3,5, 


Theorem 3,6. let F be a normal system of subsets of the 
topological measurable space (£, @. #), where all sets of # 
satisfy condition 3.10.B. For every process X continuous 
from the right in the space (F, @. #) and subject to condi- 
tion 3.10,A, a part can be generated corresponding to the 
system #, 


4, The Integral Type of Multiplicative Functional 
and the Corresponding Subprocesses 


3.16. Theorem 3,7. Let the Markov process X= (xz, 6, off, Ps,z) 
in the measurable space (E£, #) satisfy the conditions 


3.16.4, For any 0<s<t, the mapping defined by the func- 
tion x,(o) of (ix 2,91 x Ri)in (E. #)is measurable, 


Let V(u, x) (4>0, « € £) be a non-negative 9° X #-measur- 
able function and » a measure on the o-algebra #°%, The 
expression 

ag(w)==exp|— | V(a, x,)p(du Q 


(a, tf] 


now defines a multiplicative functional of process xX, pro- 
vided only that the integral in the expression is convergent 
for any 0c¢s<ct<C(w). 


Proof, The mapping defined by the function V (z, x,(w))of the 
space (/i x 2;, 87 X Ai)into (4, g°%) can be written as the pro- 
duct Va, where « is the mapping of ({x2, < Rt) into 
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(3X E, 1X B) given by 


a(z, w)={u, x,(w)}. 


In view of 3,16,A and lemma 1,3, the mapping 2 is measur- 
able, By hypothesis, the mapping V is also measurable, 
Thus Va is the measurable mapping of (/} x 2;, 2X &) in 
(fe, Wo) 5 i.e. the functionV(z, x,(o)) (e€h,0€2,) is FIX K- 
measurable, It follows from this, by lemma 1.7, that the 
function f V(u, x,)p(du) is &-measurable, so that the same 


(s, t] 
can be said of the function at, The system % thus satis- 
fies condition 3.5.4, It obviously also satisfies conditions 
3.5.B, 3.5.0 and 3.5.D. The theorem is thus proved, 


The functionals defined by (3.73) will be described as 
multiplicative functionals of the integral type. 


Let us consider the subprocess of process X corresponding 
to miltiplicative function (3.73). We have for this: 


P, {C>s+tafl*} = 
= exp | — fv. *)e (a9 (ac. Q,,,, Pea: (3.74) 
(8. 8+h] 


Suppose that, for every w€2,V(u, x,)is a function of u con- 
tinuous from the right in the interval [0,€(w))., It now 
follows from (3.74) that 


Proto >sthph'}=1—V(s, x,) p(s. s-Lal[1 +e (A) 
(a.C, Q eam Ps x): 


where e(h) +0 48S AIO. 


Hence, on the assumption that the course of the process is 
known after the instant s, the probability that it will be 
cut off after a time (s, s+ A} is equal to V(s, x,)u(s, s+A] 
to an accuracy of higher order infinitesimals, 


3.17. We now suppose that measure p coincides with the 
Lebesgue measure *) and investigate the connexion between 
the transition functions of process X and its subprocess 


*)The case when » is absolutely continuous with respect to 


the Lebesgue measure with a density of p(x) reduces to the 
case of Lebesgue measure by replacing V by PV, 
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corresponding to functional (3.73) 


Theorem 3,8, Let X= (x4, C, off3, Ps. 2) be a Markov process 
satisfying condition 3.16,A, and let V(u, x) (u>0, xE€£) be 
a non-negative*) #°X @-measurable function. We put 


t 
as (w) == exp fv (u, Xy) ca| (w € &,). (3.75) 


If the integral in (3.75) is convergent for any0<s<t<C(w) 
the function P(s, x; t,T)=M, ,[2%x,(*,)] is connected with the 


function P(s, x; t, T)=P. 2 (x,€T) by the integral equation 


t 
Ps, bee N+f f Pe. x; u, dy)V (a, y) Pa, y; t, ydae= (3.76) 
s E 
= P(s, x; El) 
Proof, For any ¢>0 and any w€Q, fv@ X,) » and conse- 


quently «§ also, is an absolutely continuous function of s, 


Therefore 


t 
fe du =1— f V(u, X,,) af du. 
§ 


On multiplying both sides by Yp(*,) and passing to the mathe- 
matical expectations, we have 


P(s, pe aan 8 


t (3.17) 
==P(s, x; t, DM, =| [Vw %,,) Cigen (*,) au]. 


*)The reader may observe on running through the proof that 


our proposition still holds when the function V(u, x) takes 
complex values of any kind, provided only that, for all «xeéE 
and 0<s<t 


Ms, 2| 02 |< Ki <0, 
t 
S| fre x; u, dy)| V-(a, 9 | da Kaeo 
E 
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By Fubini's theorem (see 1,4.C), the last term is equal to 


J Mg, o[V (2. %,) ayy (x;,)] du. (3,78) 


In view of (2.10) and lemma 1,6: 


Mg, x [V (4 i *,,) ar dr (*2)] i 
= Msg, 2{V (4 *,) M, 2, [atx (*,)]} = 
== Me 2[V(u, x,) Pu, x,; t, DT) = 
= [ Pos. x; u, dy)V(u, y)P(u, y; t, TD). 
E 


(3.79) 


A comparison of (3.77), (3.78) and (3.79) gives us (4.76). 


5. Stationary Subprocesses of Stationary 
Markov Processes 


3.18. Two variants of a theory of stationary subprocesses 
can be described, corresponding to the two meanings attached 
to the term "stationary Markov process" (cf, sec, 2.8). The 
first variant is discussed in secs, 3,186-3,20, the second in 
secs, 3,21-3.24, 


We start in the first variant from the idea of a stationary 
process as the Markov process (x; C, of, P,, x) for which a sys- 
tem of operators @, exists, satisfying requirements 2,5,A- 
265.0% 


Let ¥= (x1, t, of?, P,, x) be a Breonery subprocess of the 
stationary Markov process X=(x;,, [ of, P,,2) and let 6,, 0, be 
the operators corresponding to XxX, X and satisfying condi- 
tions 2.5,A-2,5.C, It may easily be seen that we now have 
(cf, the derivation of expression (2.32) on p. 51) 


B,{y-1A, T > 0} = (7-164, T > 2} (3.80) 
for every A€d\", 


On equating y7~-'A to A and 7-18,4A to 9A, we can write 
(3,80) in the following more compact form: 


8, {A, @ > 0} = (8,4, C>2} (AEA). (3,81) 
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. Let X bea stationary subprocess of the 
stationary Markov process X and let af be functions defined 
by condition 3.1,E, Then for any O0<s<t, x€E; 


6 = ag (a.c. 2. P, a): (3.82) 


Conversely, if condition (3.82) is fulfilled, there are 
stationary subprocesses in the class of equivalent subproces- 
ses corresponding to the system of %3; or more precisely, 
a canonical subprocess belonging to the class in question is 
stationary. 


Proof, a) Let X—(x;, 0, of%, Py.) be a stationary subprocess 


of the stationary process X=(x, 6, off, Ps). Let BEN. 
We have by 2.5.4, (3.81) and 2.6.C, 


6, (B,C >t—s} =0,{B, C> 0} 6, (F>t—s} = 
= (6,8, © > s} {>t} = (6,8, C> ¢}. 
It follows from this, in scesndaned with 2,.5.C, that 
P,, 2 (0,8, 0 >t} =P, ,(B, C>¢—s}. 
Using (3.3) , the last relationship can be rewritten as 
Ma, Xs, 3% = Mo, oka —s" (3.83) 
We have on the basis of 2.6.F and 2.6.D, 


Mo, aX Bt» = M, we [X5%7_6] = M, aXe, po a (3 84) 


vy f—2° 


Functions o and 9,07, are Wj-measurable, Since W}—6,0\_,¢ 
Ae S , it follows from (3.83) and (3.84) that 


f aP, (dw) = f 0.0% ,P. (du) 
A A 
for any ACW. We easily get (3.82) from this, 
b) Let X—=(x;, % of2, Pz) be a canonical subprocess of X 
and let functions a? satisfy conditions 3,1.E and (3,82), 


We observe that 


dV = AN X B 0, +00)» 


where #0, +0) is the s-algebra in the space (0, +-co], genera- 
ted by all intervals, Every set A€¢f** can be written 
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uniquely in the form 


A= A, Xi, 
YU... ; (3.85) 
where A,€of*, We put 
0,A= U 0A, XK A+2). (3.86) 
O<Ag?+00 


In particular, for any Cé€.\*, h>O0s 


8, {C. C> h] = 8, {[C. C > A] X (A, col} = (3,87) 
=[6,C, C>t+h] X (t+ Ah, co] = (8,C, T>¢+ A}. 
The operators 6, are easily seen to satisfy conditions 
2.5.4-2.5.B. We shall prove that they also satisfy 2.5.C. 
If 
A=(C.T>h}) (CEM, a> 0), (3.88) 


we have by (3.87), (3.3), 2.5.C and (3.82), 
~ ~ ~ ~ ¥ 
= M,, 294 [Xc%] = Mo, 2 [Xo%] = (3.89) 
=P, 2 {C. c> h} = Po (A). 
The sets of type (3.88) form ax-system @. On the other 
hand, the class ¥ of all sets A for which condition 2,5,0 
is fulfilled is a \-system in the space @,. In accordance 


with (3.89),¢2@. By lemmal.1, ¥ 2s(@)=\. This proves 
the theoren, 


3.19. Theorem 3,10. Let Y=(x;,, 0 cH#i, Ps,2) be the stationary 
subprocess of the stationary process *=(%, 6. off, Piz) 
corresponding to the multiplicative functional a == |ax3) . The 
random variables §& defined by (3.38) now satisfy the rela- 
tionship 


fs== s+ 8,8 (a,c, g,, P, a): (3.90) 


If § is any system of random variables satisfying conditions 
3.7.A-3.7.C and (3.90), the canonical subprocess correspond- 
ing to the multiplicative functional 


= Xe>e (3.91) 


is a stationary subprocess, 


86 THEORY OF MARKOVY PROCESSES 


Proof, We have for every ,r>0, 
{fo<r}S {ar = 0} 
and therefore 
(Bsbo <r} & (O50 —= 0} & {asy.—= 0} U (O,ar + aszr}. 


On the other hand, 


{Ss—s>r}= (i, > str} & {a54, > 0}. 
Hence 
{a —s > 1 > Oko} & (Bc 34 oS 47} 


Ps, 2 {§; —s >r > 8,6} =0. (3.92) 


If A is the set of all non-negative rational numbers, it 
is obvious that 


{Bs —s > 6,6} = UJ (t. —s >r > 4,6}. 


rea 
It thus follows from (3.92) that 


Paihia Ss > Osho) =O: (3.93) 
It may be proved in the same way that 


Ps, a {ts —S < O55} = 0. (3.94) 
We get (3.90) from (3.93) and (3.94). 


Moreover, if the quantities & satisfy conditions 3.7.A- 
3.7.C, expression (3.91) defines, in accordance with sec, 
3.7, a maltiplicative functional of process X. If (3.90) 
is satisfied in addition, we have 

0 
Dect ee Py <2 4a, (3, > t-s) Xe. >= ay : 
(a,c, 2, Ps, a)s 
i.e. condition (3,82) is fulfilled. It follows from this, 


by theorem 3.9, that the canonical subprocess corresponding 
to functional (3,91) is stationary, 
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3.20. We shali consider some examples, 


3.20,1, let [ be a subset of the measurable space (£, g) 
admissible for the stationary Markov process x, and let the 
random variables §,=#,(f) be defined by expression (3.49), 
We have 


{& >t}=U N [*,ET}. 


2 O<ucte— 


Hence 
{8,6 >t} =U a {y.e€T} = (> s+}. 


n 1 


The random variables §,=—§,(I) thus satisfy condition (3,90) 
and, in accordance with theorem 3,10, we can in fact assume 
that the subprocess of xX corresponding to the admissible set 
T is stationary, 


We leave it to the reader to show that the same situation 
applies for any part of a stationary process xX, 


3.20.2, Let a system ¥¢ of subsets of the phase space be 
admissible for a stationary Markov process x. Condition 
(3.90) is easily show to be fulfilled for the quantities 
defined by expression (3.58). A subprocess of X correspond- 
ing to the admissible system can therefore be chosen such 
that it is stationary. The same thing can be said as regards 
the part of a stationary process corresponding to some sys- 
tem of subsets ¥, 


3.20.3. We take a stationary Markov process X=(x;, , off, 
Ps,z) satisfying condition 3,16,A, and let a= {az}be the 
multiplicative functional of the integral type defined by 
formula (3.73). If » coincides with the Lebesgue measure, 
whilst the function V(uz, x) is independent of u, we have 


oy = 0,27». (3 95) 
For suppose we put 


t—8 


n= f V(x,) du 
Q 


and write L, for the set of all #4 -measurable p-summable 
functions in the interval [0, t—s] , for which 
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t-8& 


f 0 (uz) dua. 


We have 


a=aJ= UY TV Ve=e~} 


and by 2.5.4, 2.6.E and 2,6,G, 


{8.7 =a} = U a LV (Xu4s) = 9 (4) } = 
gel, 0Suct-s (5 96) 
8 t e 
={f V(xy.,)du =o}=| [vaoaomel, 


It follows from (3,96), in accordance with 2.6.B, that 
t 


bn { V(x) dv. (3.97) 


We get (3.95) by comparing (3.73), (3.97) and 2.6.5, 


We conclude from relationship (3.95), on the basis of 
theorem 3.9, that the functional a corresponds to a station- 
ary subprocess of process X, 


3.21. We now base our remarks on the concept of a stationary 
Markov process as a system(x,; {, of;, P,, 6, satisfying conditions 
2,8,4-2.8.H, The theory of subprocesses admits &° number 
of important simplifications from this point of view, 


Let X=(xp © of, P,, 9,) and X=(x, t, of, P,, 0, be two station- 
ary Markov processes. in the same phase space (EF, #) and let 
Q and © be the spaces of elementary events of X and X 
respectively. We shall speak of X as a subprocess of x if 
there exists a mapping 1: @-+Q2 such that: 


3.21.4, TiOa)<tiy@)] @E89). 
3.21.3, x,(w)=-x,[7(o)] (6EB, 0<t <F(0)). 
3,21,C, If A€o&#,, then {1-14, T>t}€ of, 


~ 


3.21.D, of > 7-1o40 and P2(-!A) =P, (A) for AC off, 
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3,21.B, P,{8>t|1-'0M} =at[;(0)] (a.c. 7-12, B,), 
where a,(w) is an o’,-measurable function, 


On equating as usual? (7 (w) }(@€ 8) With 9(w) (wEQ) , 
with 4,1 ={y"'A}with #¥ —{A!, we can rewrite the: ern 
tions as follows: 


oo 


Bele. Ce. 

3.21.B°, =x, 0<t<?). 

3.21.0°, If A€od, then {4, T>2}€of,. 
3,21.D°, ch Dos and P,(A)=P,(A) for AC of. 


3.21.8’, Pr{f>tlo#}—=a, (2.c,2,, P,), 
where a, is an of",-measurable function, 


If &, denotes the class of all sets A€of# such that, for 
some BE ol, P,(A\ AB) = P,(B\ AB)=0for all x€E, condition 
3.21,H can be put in the following form: 


3.21.B°", Pa{C>tlom}—= (2.0, 2, P,), 
where a, is some #,-measurable function *), 


We remark that, if (x, ¢, of, P,.) is a stationary subpro- 
cess of the stationary Markov process (x; (, of, Pz) (in the 
sense of sec, 3.18) and if the operators 0, 6, are so chosen 
that conditions 2,°,A4-2,5.0 are fulfilled, Ge Toft, Poa 4) 
is a subprocess of the process (x, € of, Po, os eee © the « con- 
verse true? Or more precisely, can we assert that, ahha 
(x, ©, of, P,, 8, is a subprocess of X= = (xp 6, ene Pi» Ss cane 
exist Stationary Markov processes X = (x, © off, P,, -) and 

X’ = (xp 0, Mf, Pe,z) such that X4.¥, xX ‘and xX isa 
stationary subprocess of xX’ in the sense of sec, 3,18? This 
question can be answered in the affirmative at least if it 
is assumed that P,{2>0}—1 for all x€F,. The proof is 
omitted since we shall make no use of the proposition, 


3.22. We shall describe the subprocess ¥=(x,, 0, of,, P,. 0;) 5 
of process X=(x,, €, of; P,, 6) as a canonical subprocess if: 


*)The whole of oe Ragen! still holds good if we put 


R= NNN, ; where the definition of W,,, in terms 
of WN; is the ‘Bame as that of WN, in terms of oN? (see foot- 


note on page 55), 
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3.22.4, The space of elementary events © of process X is 
equal to 2X], where 2 is the space of elementary events of 
process X and /J=[0, +-co]3 the mapping 1:82 is given 
by 

t(@; =o WE, XEN. 


3.22.B, Cw, ))=min[C(o), XI. 
4,22,0. X4(o, )=x,(o) for 0ct<@ iw, 2). 


3,22.D. of,consists of all sets of the form A(t, co] where 
AE of, ® 


3,,22.E, is the c-algebra in space § generated by sets 
{A, & >t}=[A, 0 > t]X (4, 0] (AE 0f9,t > 0). 


We shall speak of the system of functions @(o) ¢2>0,w€@,) 
as defining a multiplicative functional of Markov process 
X= (x, 0, of, P, 6) ifs 


3.22.0, % is &,-measurable, 

3.22.8, %9em are (A.C. Merz Pz) (8, t>0, x EE). 
3.22.7, 0S a<1 ¢D>0, EQ). 

5,22.8., hh , %,(O) = a(o) EDO, o€ &,). 


Two Pen multiplicative functionals ¢ and « will be 
said to be equivalent if, for any ‘30, x€E; 


=o, (2.0. 2, P,). 


We shall say that the subprocess x= (x, G As, Ps 8) of 
X= (x; ©, of, P,, 6, corresponds to the multiplicative func- 
tional a= {a,}-if functions a, satisfy condition 3.21.5’. 


Theorem 3,11. A one-to-one correspondence holds between 
all the classes of equivalent subprocesses of a stationary 
Markov process X= (x, 0M, P,, 6,) and all the classes of 
equivalent multiplicative functionals of xX. Each class of 
equivalent subprocesses contains one and only one canonical 
process ¥ and consists of all the stationary processes sub- 
ordinate to X . The transition function of the process cor- 
responding to the functional a= {a,}is equal to 


P(t, x, T) = MoXp (*) (3.98) 
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Proof, a) It may be seen by repeating the arguments of 
lemma 3.3 with obvious modifications that, if X= (x, 2, My 
P.. 6) is a subprocess of the stationary process , X, the 
functions a, defined by condition 3,.21.E satisfy the relation- 


ships 
0<a<l (a,c, 2). Pz); (3 99) 


yen (840. 2, P,) (3,100) 


together with relationship 3.22.8, We put 


Qe = {ap byt # Aprq} U {ap < O} U {ap > 1} U 
U {8,07 <0} U (8,0, > 1}. 
It follows from properties 3,22,8 and (3.99) that P,(Qp, 


g)=0 for any p,¢>0 xC€E. Let Q denote the sum of sets 
Qpq Over all rational » and g, Obviously, P,(Q)=—0 for all 


xCE ., 


If w¢€Q, for any rational p,q of [0,f(w)) we have the 
inequality 


0< ap +9(0) <ap(1). 


Thus, for any ¢€ [0,%(w)), @ limit exists for % when p tends 
to ¢ taking rational values greater than t, We put. 


oe 1p (~), if wé€&Q, 
a, (w) = 
es if w€Q. 
We have for any x«€E, in view of (3.100), 
P.. {%, = a} = (); (3,101) 


The function a, (w) is obviously N°-measurable; and by 
(3.101), it is &,-measurable, It follows also from (3,101) 
that functions a, Satisfy conditions 3,22,8 and 3,21,E°", 


a 


They evidently also satisfy conditions 3.22.7 and 3,22.6, 


b) Let a={a,} be any multiplicative functional of process 
xX, We shall show that there is a canonical subprocess X= 
(Xp F, A,, P o 6,)of X corresponding to «a. We find @, f Xp, 
oh,, ofoby means of conditions 3,22,A-3,22.E and associate 
with each w€2 the probability measure a,(w), on the a= 
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algebra #1, given by the requirement that 


=| a%(o), if t<C(w), 
ahel()=) 9 ap t >C(w) 


(the existence and uniqueness of this measure follows from 
properties 3.22.1 -3.22.8 of the functions a, combined with 
the results of sec, 1,11.), We put for each A€of# and each 
wEQ, 


= {A: (mw, AEA}, 
a, (w) = a4 (), 


P(A) = M,,@,- 


In particular, for any CEc#, 
P,(C, T>t}=P, (IC, >t] X(t, col} =Maxo%- (3,102) 


We define the operators 6, by the same formulae as in the 
proof of theorem 3,9 [see (3, 85)- (3. 86)]. The reader will 
easily verify that the system (Xe. . hn. P.. 6) satisfies con- 
ditions 2,8,A-2.8.H and 3,21,.A-3,21.H, It therefore defines 
the subprocess of X corresponding to the functional a, 


c) The fact that (3.98) holds may be seen by repeating the 
argument of the proof of lemma 3,1, It follows from (3.98) 
that subprocesses corresponding to equivalent functionals are 
mutually equivalent, On the other hand, our arguments when 
proving lemma 3,2 show that, if two subprocesses of a process 
X are equivalent, they correspond to equivalent functionals, 
Finally, an almost word for word repetition of the proof of 
theorem 3,1 shows that each class of equivalent subprocesses 
of xX contains one and only one canonical subprocess X and 
consist of all the subprocesses subordinate to X, 


The proof of the theorem is now complete, 


3.23. Theorem 3,12, Given the stationary Markov process 

= (x; 0, of, Pz, 94), “let a=={a,} be : multiplicative functional 
of X, where a,$,o,—4,,;for any s, f>0. w€Q,,, (a strengthened 
form of requirement 3.22.8). We ihe 


toa | eO=9 (w € 2), 


(w € &) (3,103) 
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(if o(w)>0 for all0<t<C(w), we put §(w)=2(w)), The 
function §&(w) has the following properties: 


325A, OCE(W)<l(w) (wE Q). 
3.23.B, For every t>0 {:>2}€&,. 


3.23.C, For any s>0 
{i >s} & {65—=§—s}. 
If ¢(w) is any function subject to conditions 3,23 ,A-3.23.C, 
the formula 2,=—%:,, yields a Pop eet functional of 
X , and the subprocess X= (x, oM,, P,, §,) corresponding to 
this functional is defined by the formlae 


Fw)=F()  (wEQ) 

X,(w)=%,(%) WER, 0<t <kW)) 

oft = of, [E> ty (3.104) 
Mh = oh, P, =P, 

GA={04, E>t} (AEM) 


(the space of elementary events for X is the same as for X), 


Proof, The function §& defined by (3,103), clearly satis- 
fies inequality 3.23.4. The equation 
{E> t} = {a > 0}. (3,105) 
yields 3.23,B. Furthermore, by 3.22.8 and (3.105), 
{E> s, 6.5 >t} = [a, > 0, 0,0, > 0} = 
= {a0 > 0} = {x,4, > 0) = >s+74}, 


whence we easily obtain 3,.23.C. 


Now let = be any function subject to conditions 3,23,A- 
3.23.C, Clearly, %:=%:,, now satisfies conditions 3,22.¢, 
3,22.7 and 3,22,5, Moreover 

0,950, = Xe edekes i: Zi>s, 08> = 
= Kiva, t-s>t— At>set— Mout? 
so that 3,22,3 is also satisfied, 


Finally, let & Tae conditions 3,25.A-3.235.C. The sys- 
tem (x, © of, P,, 3,) given by formulae (3. 104) will obviously 
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possess properties 2,8,4A-2,8,H and 2,8.G-2,.8.H, The series 
of equalities 


PLGA of, =P, (8,4. 5 >t] oh) = £25 Ps (Palm) = 
= 42> Pe, (A= Pz, (asc. 0,5 P.,) 


shows that it also possesses property 2.8.F, The system 
thus defines a stationary Markov process, It may readily 
be seen that X and X are connected by relationships 3.21,A- 
3,21.E (where y(w)=w). Hence X is the subprocess of X. 
corresponding to the functional {a,}. 


3.24. Let X= (x, % %, P, %)be a stationary Markov process 
in the measurable space (£, #), such that &,¢ of#,, and let 
TCE. Im accordance with (3.49) we put 


E(D) = inf (t: x,€ TI}. 


We shall call the set IL admissible for the process X if 
E(T) satisfies condition 3.23.B, Conditions 3,23.A and 

3.23.C are easily seen to be also fulfilled in this case, 
Formulae (3.104) thus yield a subprocess of ¥. We shall 


refer to it as the subprocess corresponding to the admissi- 
ble set I*), 


The transition function of this subprocess is given by 
Bit, x, G)=P,(x,EG, EP) > 2}. (3.106) 


We now consider the system (x, ©, of, P,, 6,) defined by 
formula (3,104), the measure P,, being considered for x€TI 
only, This system evidently yields a stationary Markov pro- 
cess in the measurable space (I. @I[T}])s. We shall describe 
it as the part of process X in the admissible set I **), 


Theorem 3,13, Let £,€ 4% be a sequence of sets in the 
measurable space (EF, @) such that £&,f& . Suppose, further, 
that for every 1 a stationary Markov process x = {x 7, 
AM”, p, aMlis given in the measurable space (£,, #{E,]}), X™ 


*)Tt must be emphasised that X and I uniquely define the 


subprocess X of X corresponding to the admissible set I’, the 
space of elementary events for ¥ being the same as for xX. 
**) As distinct from the general case discussed in sec.3.9- 
3.10, a part of a stationary process X=(«;, %, M2, Pr, 02) 

can be generated in every admissible subset, 
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being the part of X+) in the set £,, Then a stationary 

Markov process X =(x,, {, of;,P,. §,) exists in the space (E, #) 

such that (C(w)= liml™(w) and x) is the part of X¥ in the 
n>=> 


set E,. 
Proof, Since X™ is the part of X+!) in the set £,, we 

have in accordance with (3,104), 

CO (w) OF) (w), 

xf (wo) =x" w) for 0<t <0 (w), 
‘a Arr 1) 2”) (w) > 4] 
(off) —= (offo)"*") and p® — pty for xCE,, 
A= [674 > 2. 


Therefore the limit exists: 
€(w) = lim 6 (w) 
n>c 


and a unique meaning, independent of the choice of n, attaches 
to the expressions 


“(=x if oct<e™@ | 
oh, = AY [5 (w) > f] (3,107) 
of? = (of), p, =p”, if x€E, 
O.4=={004,o>¢}, ap ACN. 


The system (x; ¢, of, P., 9,) is easily seen to satisfy condi- 
tions 2,8,4-2,8.H, so that it defines a stationary Markov 
process X, For this process, the instant ¢(E,) of departure 
of the trajectory from set £, coincides with ¢”, and the 
part of X in set £, coincides with X™) by virtue of 


(35207) *). 


All the definitions and propositions relating to subproces- 
ses and parts of stationary processes corresponding to admis- 
Sible sets are easily carried over to subprocesses and parts 
corresponding to admissible s:stems of sets, 


*)It may be remarked that ¢ coincides with the instant of 


first departure of the trajectory of X¥ from the sequence {£,}, 


CHAPTER 4 


THE CONSTRUCTION OF MARKOV PROCESSES 
WITH GIVEN TRANSITION FUNCTIONS 


1, Definition of Transition Function, Examples, 


4.1. We take any measurable space (EF, #). The function 
P(s, x: DO<s<t, x€E,TE#)is said to be a transition 
function if the following conditions are satisfied: 


4.1.4. P(s, x: tT) is a measure, (As a function of the set 
r.) 


4.1.B, P(s x: 4, is a #-measurable function of x, 


4.1.C, P(s, “iE, n< i. 
4,1.D, P(s. x; 5 EN x)=0. 


A.1.B, Pls, x: u, N= f Po. x; t, dy)Pt yi uw NOSsct<n), 
Ez 


We shall describe the transition function P(s, x; #, T) as 
normal if P(s, x;s,F)=1for all s>0, x€E, and as complete 
if Pis, x; t, H=ifor t>s>0, «€E.- 


There is a corresponding transition function for every 
Markov process: 


P(s, x; t, T)=Pe, 2 {ET} 


(properties 4,1.A-4.1.D follow from 2.1.C, 2.1.D, 2.1.5; 
property 4,1.E follows from(2.12)). If the Markov process 
is non-cut-off, the corresponding transition function is 
complete. 


The aim of the present chapter is to investigate under 


what conditions does the transition function P(s, x; 4, I 
correspond to some Markov process. (The connexion between 
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the various Markov processes having the same transition func- 
tions was discussed in article 3 of Chapter 2.) 


4.2. We shall consider some examples of transition func- 
tions, 


4.2.1. let £& be a numerical straight line, # the class 
of all Borel subsets of EF, and v an arbitrary constant, The 
expression 


P(s, x5 t, =y,[x+u(¢—s) 


defines a complete transition function of (F.¢). We shall 
describe this function as corresponding to a determinate 


motion with velocity v. 


4.2.2, Let E be the set of all natural numbers and # 
the class of all subsets of this set, and let pi;(s. Oi, j=], 
2,....%...; 0 S<t)be a system of functions satisfying the 
following conditions: 


4.2.2h. Pigs ODO. 
4.2.2.3. drut N<i. 
4.2.2.0. piyz(s,s)=Ofor i+j, 
4.2.2.D. Sruls £) Pye (t, 4) = pix (S, 2) (S<St<a). 
The expression 
P(s, x; t, 1) = 2 Pew (s. f) (EE s<teTeE#) (4.1) 


now defines a transition function, and it may easily be seen 
that all the transition functions in the space F=({I, 2,..., 
n,...}can be got by this method, The necessary and sufficient 
condition for the transition function to be normal is that 
Pu(S,s)==1 . The condition for completeness is given by 


D Piz (s» f= 1. 
j=l 


Everything that has been said is applicable also to the 
case when E£ consists of a finite number of points {1, 2,..., 
n}, 


4.2.3. Let E be the n-dimensional Euclidean space &” 
and # the o~algebra generated by all open sets of this space, 
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We write for each x€F, TES, 


| (2s (¢—s)) 2 [exp [— 3] dy 
P(s, x; t, y= { ; “a (4.2) 
for 0<s<4, 


ee for 0<s=zt, 


where (y—-x)? denotes the scalar square of the vector y—vx 

and integration is carried outwith respect to the usual Lebes- 
gue measure on Rk”, It may easily be verified that the function 
P(s, x; t+, defined by this formula satisfies all the conditions 
4,1,A-4,.1,.E and is consequently a transition function, This 
type of transition function is complete and is known as a 
Wiener transition function, 


4.2.4, Let E=(0, o) and @ be the c-algebra in the space 
E generated by all intervals, We shall consider a function 
closely connected with the one-dimensional Wiener function, 


Pi, 5t,D0=— 
1 (y — x)? (y-+x)? 
Ve G=5) { {er [ea] -exe [ES] } (4.3) 
pa for 0<s<i, 
Xp (x) for 0<s=t. 


This function clearly satisfies conditions 4,1.A-4.1,.D.:‘A 
certain amount of calculation shows that it satisfies 4,1.E, 
The transition function (4,3) is normal but not complete, 


4.2.5. We keep the same values for E and # as in sec, 
4.2.4 and put 


P(s, x; 2, = 
| i ff exp [FS Hex [-Se]}4 
V 32 (f—s) (¢—s) PL 2G=s) I 
a r (4.4) 
for0<s<t#, 
| Xp (*) for 0<s=t, 


It is easy to verify that P(s, x; t, T)is a complete transition 
function, 
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4.2.6, Let (£, #) be any measurable space and I(x, 1) (x€E, 
ré #) a function satisfying the conditions: 


a) U(x, Tis a probability measure on # for any x€F; 
b) U(@, ryis a @-measurable function in — for any TE€@. 


We define functions II, (x, Tyas follows: 
II (x, YT) = x, (*), 
Hn (% TY= f(x dy) Trad) (> 1). 
B 


It may readily be seen that: 
1) 0<I,(«, I<! for any 1>0, «€E, TES; 


2) Dnem(* T)= f 1, (x, dy) Tp (y. TD). 
E 
We put 


; i — an (tf — 5)% ‘ 
Piss et D=y ASS e-2t-O (x, Ty, (4.5) 


n=0 ‘ 


where a is a positive constant. In view of 1), the series 
on the right-hand side of (4.5) is convergent, whilst in 
view of 2),P(s, x; t, M)satisfies condition 4.1.E, It is easily 
seen to satisfy conditions 4,1,.A4-4.1.D also, We thus have a 
transition function which is clearly complete, It will be 
described as a Poisson transition function, 


2, The Construction of a Markov Process with Given 
Transition Function 


4.3, Theorem 4,1. Let (£, @. #) be a s-compact topological 
measurable space satisfying condition 1.9,.A, Every complete 
transition function P(s, x; 1,T) O<s<t, x€&, TE), will 
now correspond to some non-cut-off Markov process in the 
phase space (£, @, #). 


Proof, We shall construct the non-cut-off Markov process 
X in its canonical form, i.e, we take E’(/=[0, co)) as the 
space of elementary events, put *;(7)=¢9() @€/, oC EF) and 
write oN’ for the c-algebra in the space E£! generated by 
sets {9:¢(u)ET} (u>s, TC #) and WN: for the s-algebra gener- 
ated by sets (9: ¢(ET} (s<u<ct,T€ #) (see sec. 2.13). We 
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still have to define the probability measures P, , in the s- 
algebras W*; this is done by using theorem 1,2, 


We first of all use (2,15) to construct in accordance with 
the transition function P(s,x;t,T, the functions 


PS 6) ti Tis ence Tye Te) 
(XEE, sctck<...<th, Th...) Mr€ #). 
We put, for any 4,..., 4,€[s8, co), 
®.,..., mee stk pee Si A es Dae eas tins rT.) 
where 4i,<4i,<...<¢h,, and (i, .--: i,)is some permutation of 
the numbers 1, 2,..., 2. The functions %:,....4, are easily 


seen to satisfy all the conditions 1,12,A-1,12.C, We use 
theorem 1,2 by putting T—=[0, co), T=(s,co). The theorem 
states that a probability measure P satisfying condition 
(1.28) exists on the s-algebra Wy=d’, The measure P 
depends on parameters s and x, and we write it as P,,. We 
have for any x<€E, 0<sct,<...<t, Ty.... ThE 8: 


Pe 2 (xe, ET, or x4 € r,,} = P(s, x; tT, «+s t,; Ty): 


The system (x, oN, P, .) Obviously satisfies conditions 2.1.A- 
2.1.8, Furthermore, the functions P(s, x; t,, Ty, ..., ¢,, T,) 
satisfy expression (2.14), and by lemma 2,1, condition 2.1.F 
is fulfilled, The system (x, ot, Ps,,) thus defines a non-cut- 
off Markov process, The transition function of this process 
obviously coincides with P(s, x; #,T). 


4.4, Theorem 4,2, Let (E. C, #) be a c-compact topological 
measurable space subject to condition 1.9.4. Every normal 
transition function P(s, x; ¢, T) will now correspond to some 
Markov process, We can assert, furthermore, that the function 
corresponds to some part of a non-cut-off Markov process, 


Proof, We write E for the set obtained from & by the 
addition of a single point ec, let AcE. We put A€@ if 
ANEEG, and AC¥ if AMVEEC#, It is easy to see that 
(E, @, @) is a c-compact topological measurable space satis- 
fying condition 1,9,A, We define the function P(s, x; #,T) in 
the space (E, @, #) as follows: 

P(s, x; 2; hn= 
P(s, x; 4, PE)+-y,(@)[1—P(s, x; t, BE). (4.6) 
={ if *#a, 
Xp (2). if *x=a. 
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The function P— is easily seen to satisfy conditions 4,1,A- 
4,1.5, so that it is a transition function, This transition 
function is complete, and by theorem 4,1, a non-cut-off 
Markov process xX exists in the phase space (E. @, #) corres- 
ponding to P, Using lemma 2.4, an equivalent process X of 
the non-cut-off process X can be constructed, for which the 
set E is inaccessible from a, Since the function P(s,x; tT) 
is normal, F satisfies condition 3,8.B, In accordance with 
theoren 3, A, we can generate a part of process X on the set 
E, Taking into account (3.55) and (4.6), the transition 
function of the 'part' is seen to coincide with P(s,x:tT). 


3, Stationary Transition Functions and the Corresponding 
Stationary Markov Processes 


4.5. The transition function P(s, x;#,T)is said to be station- 
ary if it depends only on the difference t—s, We shall 
write in this case P(t—s, x,T) instead of P(s,x;#,P). 


The functions constructed in 4.4.1, 4.2.3-4.2.6 may be 
mentioned as examples of stationary transition functions, 
The functions of 4,2,2 are stationary when, and only when, 


the functions p;;(s, f) depend only on the difference {t—s, 
We shall write p;;({—s)in this case instead of Pi;(s.f), 


If we take into account conditions 4,1,.A4-4,1.E, we can say 
that the function P(4,~,T) (A>0, «€F, TEC #) is a stationary 
transition function when, and only when: 


4.5.4. P(h,x,I) is a measure, (as a function of the set 
Yr). 


4.5.B. P(h,x,T) is a #-measurable function of x, 
4.5.0. 0< P(A, xT) <1, 


4.5.D,. P(0,«, EN x)=0. 


4.5-Be Pith x, T= f Ply x dy) Pll 9 TD (hy hy, > 0). 
£ 


The condition for a stationary transition function to be 
normal is 


P(0,x,E)=1 (x€2). (4.7) 
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The condition for completeness is 
Pile) eal (¢>0, x€ E). (4.8) 


In accordance with theorem 2,10, the class of equivalent 
Markov processes contains stationary processes when and only 
when the transition function corresponding to this class is 
stationary. 


4.6. We shall indicate a method by which any transition 
function may be associated with a stationary transition func- 
tion in a rather more complicated phase space, In view of 
the connexion between transition functions and Markov pro- 
cesses, this method enables us to reduce a discussion of non- 
stationary to a discussion of stationary processes, 


Given the transfer function P(s,x;t,.T)/(O<¢s<t, x«€T TE#) 
let (E, g)>e a measurable space, where F—/XF, #=@& xXx @ 
(= [0, co) whilst #¢ denotes the c-algebra in space /) génera- 
ted by all intervals), and let the function P(z, y, B) (h>0, 
yCE,BE ) de defined by 


P(h, Vy, B) = P(t, as tt-h, Y), (4.9) 


where y=(f, x) and the set I is defined by the condition zéP 
if ¢+4, 2)€8, (It follows from lemma 1.4 that TC #, so 
that (4,9) has a meaning for any h>O0 and any BE.) It 
follows from 4,1,4-4.1.E that the function P(A, y, B) satisfies 
4.5.4, 4.5.0, 4.5.D and 4,5.B, In order for 4,5.B also to be 
satisfied, the additional requirement is needed that P(s, x; t, T) 
as a function of s, x and ¢ should be measurable, 


CHAPTER 5 


SCRICTLY MARKOV PROCESSES 


1, Random Variables Independent of the Future 
and s-Past, Lemmas on Measurability 


5.4 Among all Markov processes we shall distinguish an 
important class for which the principle of the independence 
of the future from the past for a known present is valid in 
a strengthened form, More precisely, in the statements of 
the condition for a process to be Markov it is a matter of a 
fixed instant t (one not depending on chance); for the pro- 
cess to be considered in the present chapter, a similar con- 
dition is also satisfied for a definite type of random vari- 
able (random variables independént of the future and s-past ) : 


Let X= (xp, 0, of, Ps, 2) be any Markov process, We shall 
call the non-negative function <(w) a random variable indepen-~ 


dent of the future and S-past if: 
5alA, s<t(w)<max{s, S(w)] (WE 2). 
5.1.B. fwrt(wy<ct<l(w)}EM: (s<n. 
(Condition 5,1.B can be visualized as meaning that the 


answer to the question, which is the greater, « or ft, depends 
only on the phenomena observed in the time intervals [s, min 


(t, 0)] -) e 


By 5.1.4, (t>t} U([t<t<t}=(f>t} € oft , so that 5.1.3 
is equivalent to the following condition: 


5.1.3‘, {w:t(w)>t}€ fi (s<t). 

We put 2 = {w:t(w) <C(w)}. 

The subsets AGQ@., such that for any t>s,{A,.t<t<f}eE 
oi , may easily be seen to form a ‘c-algebra in the space Q.. 


We shall denote this c~algebra by of. It is natural to 
regard o#% as the class of all events observed during the 
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fime interval [s, +]. 


If the function « satisfies 5.1.A and takes only a finite 
or denumerable set of different values f/f, 4%, ...€[s, oo) in 
Q., it is a random variable independent of the future and 
the s-past when and only when (t=, <C}€o#:, for k=1, 2,... 
With this, A€o#? when and only when (4,c= 4,<C}€oMj, for 
all k. 


An example of a random variable not depending on the futute 
and s-past is given by the function 


| t,if ‘C(w)>f, 
t(m)= 7S), if s<Cw<é, 
s,if Cw<s. 


For this function, 29.—&, . Another example is the function 
t(w) = max [C(w), s] (ror which @.=—g@ ). Further examples are 
provided by the systems of random variables § satisfying 
conditions 2.7.4-3.7.C, on the assumption that Rico#i*) ; 
and in particular, by the instants of first departure after 
s of the trajectory from the admissible set I (see sec.3,8) 
or from the admissible system of sets ¥ (sec. 3.11). 


The function §(») defined in ©, is measurable with respect 
to of", when and only when for any t>s the function induced 
from § in the set [t<¢<C} is measurable with respect to 
oft. The measurability of the function § with respect to 
the o-algebra off may be pictured as meaning that § depends 
only on phenomena which are observed during the time [s,t], 


It may be remarked that the function t(w) satisfying condi- 
tion 5.1.4 is a random variable that does not depend on the 
future or s-past when and only when its restriction to the 
set © is measurable with respect to of. 


Lemma 5,1. Lett be a random magnitude indepéndent of the 
future and s-past for the Markov process X, and let Q’€ off. 
Moreover, let the function 7()(wE€ 2.) be of#f-measurable and 
satisfy the inequality 7 >:. Then the function 

| : [ n(w) for o€ 2, = (Q’, 7< 4}. 
i oe max [s, &()] for o€ 2. 


*)This assumption does not seriously affect the generality 


since by sec,2.2 we can always extend the c-algebra { to 
om = Hi without thereby transgressing requirements 2,1.A-2.1.E, 
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represents a random variable independent of the future and 
s-past, and AQ,» of. for any AC of’. 


Proof, Since 72%, we have for any t>s, 
{n<te{t<t} 


and [M*®<E<O SH (QL ttc (nt cSt KU E off. 


The first assertion of the lemma is proved, Furthermore, if 
Ato , we have 


{AQ SEO SH lA, ttc lVy*<t<l} CoM}, 


which proves the lemma, 


5.2, We put for brevity ;=[s, t], ’=[s. oo), 4,= [0. 4] and write 
#, # and #, for the s-algebras generated by all intervals 
in f, P and /, respectively, 


The function x(u, w)=-x,(w) induces a mapping of the measur- 
able space (XxX 2, BX oft’) into the measurable space (E, #). 
The Markov process is said to be measurable if this mapping 
is measurable for any 0<s<t. 


Lemma 5,2. Let X=(x;, ¢. MPs.) be a measurable Markov 
process and t(#) a random variable independent of the future 
and s-past, Then the function 8(w)=-«[t(w), ©) defines a measur- 
able mapping of (©, o#%) into (& #). 


Proof, We put C,={t<t<C}. We have to show that, for 
any t>s , the mapping of (C,; ,o#{) into(E, #) induced by 8 is 
measurable, Since t defines a measurable mapping of (C,, offs) 
into (41,41) , the expression «,(w)= {t(o), w} defines a méasur- 
able mapping of (C,, off) into (Ji xX 2, #1 X of?) (lemma 1.3). On 
the other hand, in view of the measurability of the process, 
the mapping @ of (I XQ, BX oft?) into (£. #) induced by the 
function x(¢,) is measurable, Consequently the mapping ™2: 
of (C,, of;) in (E. #) is measurable, But obviously, «2, coin- 
cides with the mapping induced by fin C,, 


Lemma 5,3, Let X=(x, C. of, Pa.) be a measurable Markov 
process and let the function 


P(s, x; t.T)=P,. {x,€T} (s€ f,, x€ E) 


be measurable for any t>0,T€ # with respect to @:x#, 
Then 
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a) whatever ther>o0 and A€W\", the function 
Ps, 2 (A) (sEl,, x € E) 
is #,X @ -measurable; 


bd) for any n>1 and any 7,,....T,€ @, the function 


P(s, x: ty Dy .--s bg Dy) = Paw (%4,ETi «++ 1, ET} 
(s< ty... tg XEE) 
is #°X # X[#°]"-measurable, 


Proof, Assertion a) can be proved simply by repeating the 
proof of lemma 2,2 with a few trivial changes, 


We turn to the proof of assertion b). We take the function 


PS, 4 LE) AS Pye (Ca). At oct. 


, = 
Berek) 1 if s>t. 


With s and x fixed, p(s, x,t) is a function of ¢ continuous 
from the right, With ¢ fixed, it is a XxX #-measurable 
function of s, x, It follows from this, by lemma 1,10, that 
o(s, x,t) , and consequently also P(s,x; t,E) is aG°xK @X g- 
measurable function of s, x and t, We complete the defini- 
tion of function yr[x;(w)] by putting xzr[x;(o)}=—0 with wEé®,, 
Now clearly, 

Psi ty Dis si sctas Ty) = 


= fx, [xXe, (o)] --- Xrn [Xt, ()] Ps, » (2). (5.1) 
2 


We select an arbitrary r>0. It follows from the measur- 
ability of the process that, for any k=1,2,...,, the func- 
tion Xr, [%, I (,€1,0€2) is gx o#’-measurable, The 
function 


Xr, [Xt,(w)] ... Xr [+s (o)] (t,, 5 ff EL, w € Q) 


is therefore measurable with respect to [9']”« c#”. In view 
of a), the function P,(A) is #,X #@-measurable for any 

A€ . Therefore by lemma 1.7, the function defined by 
(5.1) in the set s€J,, x€E,t,....t,€7% is #,xX #X1#)"- 
measurable, 


We now observe that, if Cé€ g° and A is the set of systems 
(s, x; th eees t,) for which 
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(Ge Fa eth a Siete Vis ented eC. 


we have 
a=[Ul4. 1, XEX UT) |UlA tis} U 20 UTA Ss (52) 


where r runs through all non-negative rational numbers, 


We have by what has been proved: 


{4.1,XEXUTIEC SX BXI@T SR X Bx gy". (5.3) 
We now notice that, by 2.1.H, 


P(s, x; 8. T) =, (x) P(s, x; $, E). (5.4) 


Hence with 2=1, 


{A, t= 8} = (t==5, 7, (x) P(s, x; ty, EEC} E B°X BX #, (5.5) 


and assertion b) follows from (5.2), (5.3) and (5.5). With 
n>1, we have by (2.14) and (5.4) for k=1, 2,...,23 


{A, tes) lips; S<A, ery bes Lrtts eee 


Lr, (x) P(s, x3 ty, EVP (Ss. #; ty Ty - ees Cpegs Pena. 
eee lr iey Dregs oe ey Lai T,EC}. 


If we assume that assertion b) has already been proved for 
the case n—1, it follows from this that 


{A.%=s}S BX BX [ go" (5.6) 


On combining (5.2), (5.3) and (5.6), we conclude that 
assertion b) is likewise satisfied for the case nz, 


Remark, The function P(s, x; ¢, T) is always measurable 
with respect to x, and the arguments used in proving lemma 
5.2 show that, for any measurable Markov process, the func- 
tion P(s, x; ¢,T) is measurable over the aggregate of x and ¢, 
For the stationary process X=(x,, 6, off%, Ps, 2) we have P(s; x; t, 
YT) =P(0,x,t—s,T) by theorem 2.9, Hence for any stationary 
measurable Markov process P(s, x; ¢,T) is a measurable function 
of s,x,t . 
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2, Definition of Strictly Markov Process 


5.3. Definition, The Markov process X= (x, 0. of, P,,.) in 
the phase space (FE, #) is said to be strictly Markov if it is 
Measurable and satisfies the following conditions: 


5.5.4, With any t¢>0,TC@, 
P(s, x; t, T)= Ps, 2 {x, ET} 


is a #,xX @-measurable function of s and x (¢;1is the s- 
algebra of the subsets of space /J,=— [0,7]. generated by all 
intervals contained in /,), 


5.3.B. If + is a random variable not dependent on the 
future and s-past, we have for any offz-measurable function 
7(w)>t(w) and any x€F, TE, 


Pea lx ET loMs}=P(s, x37. T) (@.0.2., Peo). (5.7) 
Condition 5.3.B can take the following form: 


5.3.B°. Whatever the x€F, s>0, TE®, the random varia- 
ble + independent of the future and s-past, and the offi - 
measurable function 17(w)>+t(w), the following equation holds 
for any A€of:: 


Ps 2 (A. x,€T)= f P(t x5 4 T) Pe, 2 (du). (5.8) 
A 


For, by definition of conditional probability, requirement 
5.3.B is equivalent to the combination of 5,3.B° with the 
requirement that P(x, x,; 4, Mbeanc#:-measurable function of o. 
It follows from condition 5.3.A by lemma 5.3.b, that the 
function P(s,x; 4,T) is @°X #X @-measurable, The of;- 
measurability of the function P(t, x; 7,T) therefore follows 
from lemmas 5,2 and 1,3, 


5.4. Later on we shall construct examples of Markov pro- 
cesses which are not strictly Markov (see sec. 6,18). Con- 
ditions 5.3.B-5,3.B° are not fulfilled for these processes, 
We now show, however, that a weakened from of the conditions 
is satisfied for any Markov process, 


Lemma 5,4. No matter what the Markov process xX, conditions 
5.3.B and 5.3.B” are satisfied if the random variables + and 
7 take only a finite or denumerable set of values in @.. 
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Proof, Let t take the values 1t,.4,..., and 7 the values 
Uy, Ug, .---Let ACM, TCH. 


= (A, t= 4} (n= uy. t=} CoM, (4 R=1,2..--), 
and by 2.1,F* we have for ti < uy ; 
Ps, 2 (Ain %, ET) = Ps a (Aine Xy, ET) = 


= f PG X45 Lp D) Ps, 2 (dw) = f Po HAN; T) Ps, x (dw). 
Ary Aix 
On summing these relationships over all pairs i, & for 
which t4;<u4,, we get equation (5.8), 


We now notice that the function %(w)=P(<, x; 7,I induces 
in the set A, the function P(t; x;,; 4 I), which is clearly 
offi, -measurable, Therefore, whatever the number a, we have 
for 4<f: 


{ Aix, p(o) <a, ee 
={P (tis 43 we T) <a t< EN Ase £< 8) € oft 


The sum of all these sets is equal to [$()<at¢t<%}. 
Hence this latter set also belongs to o#{. The function 4(w) 
is thus o#?-measurable, and (5.7) follows from (5.8), 


5.5. Condition 5.3.B or 5.3.B° for a process to be strictly 
Markov is to be regarded as a variant of the principle of 
independewe of the "future" (x,) from the "past" (of#*) for a 
known "present" (x,). We now show that this principle remains 
valid for a wider conception of "future" (any event, defined 
by the values *,,---» % where 7 ....%, are random variables 
depending only on the phenomena observed during the time 
[s.t]) « A precise formulation is given in theorem 5,1, the 
proof of which follows the same lines as the proof of theorem 
2.1 except that certain extra technical difficulties have to 
be overcome, 


We shall first prove a lemma, 


Lemma 5,5. Let X=(x, €. off, Ps,c) be a strictly Markov 
process, t a random variable independent of the future and 
S-past, 7>7 an of#{-measurable function, and (w€2., x€E) 
®(w, x)an of: X #-measurable function such that M,,,P(w, x,) 
exists, Then 
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Me, 2 {P(w. x)}o#} =p. t xy 2) 


(a,c. 2, Ps, 2): (5.9) 
where 
Poy ws ¥ V= f Plog x, (0)] Puy do). (5.10) 


Proof, We suppose first that 


Dw, )= Ho) Zp(x) (CE oA, TE B). 
Then by 1.6.F, 
Ms, 2(®(o, ,)] 3} = 75 (w) Me, 2 [Xp (x,)] i} = 


=Yo(v)Psx{x,ET] of} (a.c.2,, Ps, 2) (5-11) 
and on taking (5.7) into account, 
Mg, 2 {® (o. x,) | of?) = 7, (w) P(t, x.; 1 T) (5,12) 
(a,c. Q., Ps, 2). 
On the other hand, 
Y (Wo, es Ys V) = Xo (9) P(u, y; v, YP). (5.13) 


Equation (5.9) follows from (5.12) amd (5.13). 


Let -¥ denote the class of all functions O(w, x)@E2, xEE) 
such that Ms,2P(w, *x,) exists, The class $ of all functions 
®(w, x) for which relationships (5.9) and (5.10) are satisfied 
is evidently an .%-system, The sets CXTI form a n-system C, 
By lemma 1.2, the ¥-system 5@ contains all functions © of 7 
measurable with respect to the s-algebra.c(@)=cfi x @. The 
lemma is proved, 


5.6. Theorem 5,1. Let X=(x;,, 6, oft, Ps) be a strictly 
Markov process, < a random variable independent of the future 
and s-past andt<7,---» % a system of .f#*-measurable func- 
tions, Then for any [,....,.hMé¢g, 


Ps, x {x,, ET) eee x2, ET, | of} = 


el 
=P(t, Xs Nhs Dis sees MWnp T,,) (a.c,2., Ps. as (5 4) 
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where P(s, x; t,T,,..., ¢, T,) is given by formla (2,13), 


Proof, With n=1, (5.14) reduces to condition 5.3.3. 
We assume that (5.14) is satisfied for n—1i, and prove it 
for n, We put 


4 (©) = min (y, (w), ..., Mp (w)), 
Ay={%1 > +--+ M1 > yD Tk = 7}. 


We notice that A,Co%, +S nc. m .--. Taand 7 are measur- 
able with respect to o#f;, We fix a value of k(l<k<n), put 


. n(o) for »€2,,= [Ay 1< 4}, 
WO) = 1 max (s, %) for w€ 2, 


and denote the random variables %.---. %, considered only in 
the set 2, by 1 .-.. 7%. It follows from lemma 5,1 that 

y’ is independent of the future and s-past and 7; (w), ..., 

7 (w) (» € &,,) are o#§,-measurable, Apart from this, the in- 
equalities yi >7° are satisfied, so that by our inductive 
assumptions: 


Pa, o(Ch oMis)= Ay (a Xai The ee They Thay oo T= fs) 
== Hi, (1, Hy Nyy se) Ne-s Netty «++» Nn) (A.C. 2, P;, a): : 
where 
aa tari veee Xm ET;,-1; | 
1 ‘k—1 
“* ET par -- x? er, 
sneTine AEM Ls 
A (Gi VP Ope 3 Dy) = 
=P (a, Y3 Up Tye oo) Oper Tyags Uns Pits sees Oa—1s Tp). 
We put B={x,, ET, ..., x;,€T,} - Obviously 
nN 
B= UlAe #4, € Te Cy. 1p) 


By (5.15) and (5.17), we have for any A€of*, 


112 THEORY OF MARKOV PROCESSES 


42 
P, ~(AB) a) Pe, 2 {AAg 5, ET Cy) = 


n 
=o f FG Migs ga 
c=] {Ady 2, Te 


sect Menty Megas +e Mn) Ps, a (dw) = (5.18) 


n 
= 3) My olP:(0. 4) 141= 
2 
=F My, o {2.Ms, 21 Px (o, ,)[ of%]}, 


where 


®, (, x= 
Ly () Ap, (8) He (te 05 Ne oe Meets Meat eet (5219) 


The mapping 9.x E-—>/ defined by the function ©,(w,x) can 
be written in the product form fa, where the mappings 
a:2.xXE->[P]"xE and 8:[/}* F+f are given by the expres- 
sions 

a(o, x«) = 
= {1 (o), (©) .- +) M1 ()» Mt1() » +++ Na(w), x}, 
B(G, Uy, .- 0. Ung» HSE Ay (OU, X3 Dys 000s Ung) 


It follows from lemma 1,3 that a is the measurable mapping 
of (2.x E. of g)in (IP XE, [# "x #). In accordance 
with lemma 5.3, 6 is the measurable mapping of ([/]" x E, 
[Py K #) in ( nh, Bi) . The function ®,(w, x) is therefore 
AZ @-measurable, and we are justified in applying lemma 
5.5.- In our case | 


Y (wo, we Ys = f Pxlog, xX (0)] Pu, y (do) = 


2 


= fo x4, (0) tp, [0 (Ol Hala (os Xo (0) Ma (oo)s -+- 


vee Ne-1(o)> Ne+1 (Mo) - ++» Na (o)] Py, y (20) = 
= 4, (00) f Plt ys dz) Hy ln (oo). 2 1 (wo)s --- 
T, 


os Ne 1(Mo)> Neat ()s ---+ Nn ()]; 
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and by (5.16) and (2,14), 


Yo: Hs Ye MOY)I= Ka (o) P(e Vi 1 (Ho). Te Mr (ms Tye. 
eee Me 1 (yds Teorey Near (Wo) a (m9), T) = 
mae Fs (%)) P(u, ys Mm (wm), Ty ..., Nn (Wo), T,,). 
Hence 


p(w, % x,, =a, (0) PC ¥23 TM Ty .... My Ty). (5,20) 
On comparing (5.9) and (5,20), we find that 
Ms, v {®,. (, x,)] ots} = 
= ha, PG X33 Ty... My r,,) (a.c.Q., P,, ‘eh: (5.21) 
It follows from (5,18) and (5.21) that 
P, « (AB) = M, 2lZ4P(s Xe Ny r,, seen Tins r,)]- (5.22) 
Since the function P(t. x; 7, Ty, ..., 1, T,) is o#%-measurable 
(as follows from lemmas 5,2 and 5.3), (5.22) is equivalent 
to (5.14). 
Corollary. Let X be a strictly Markov process, If «+, 


TN +--+) NM» ---is the system of w-functions described in theorem 
5.1, and f(x,,...,*x,,...)is any @”-measurable function in the 
space E~ , such that M, f(z, +++» 1, -+-) exists, then 


Mg, 2 {f (Fay +0) X ny vo PBs] SP (x ty eee Tae ee 


) 
(a.c.2., P, ,), (5.23) 


where 
Pi Fo Ciscsatg Oye ee VS My Gh owe Segeecs)e “(5 e24) 


Proof, We write 2 for the class of all functions f for 
which M, f(xy, ---» %s,:---)exists, and $f for the class of all 
functions f for which equations (5.,23)-(5.24) hold. By 
theorem 5,1, -7 contains the characteristic functions of 
sets T,xX...XI,(#=1, 2, ...; Ty...., h€@). These sets form 
a x~system generating #°, Obviously, % is an -%-systen, 
and by lemma 1.2, 5¢ contains all the @@-measurable functions 


of 7. 


114 THEORY OF MARKOV PROCESSES 


5.7. We shall give a further important formlation of the 
principle of independence of the "future" from the "past" for 
a known "present." Some new notation is first introduced, 


We shall write 3° for the z-algebra in the space 2x /’ 
generated by sets 2x />(T>s) and 


{(w, £1): x3 (y(w) ET} 
(TE #, z() is an arbitrary #*-measurable function in the inter- 
val /° sucn that <()>¢ for all t). Evidently, J* can also 


be described as the minimal s-algebra in space 2x /* with 
respect to which the functions 


f(t, 0) =f and O(¢, 0) = x.y ()s 

are measurable, 
Lemma 5,6, If T>s and §(w) is an arbitrary o?-measurable 
function, the function yz, » (/)§(w)is 9*-measurable, All sets 


of the form 
([x.EI} XIs. T] (>T7. TE F) (5.25) 


belong to the s-algebra 3*, In conjunction with sets 2x /% 


(T>s) they generate the s-algebra j* in the case when X is 


a& process continuous from the right in the topological 
measurable space satisfying conditions 1,9,A-1.9.B. 
Proof, In accordance with lemma 1,5, 
6 (w) = f(x... Xe. +++) 
where ¢,,..., ty... >Tand f is a #”-measurable function in 


the space E°, We take any g%,-measurable function 3; (t) 
satisfying the conditions 


GO=4; for s<t<T, 


Obviously, 
Ats, T} 2) ; () = Tis, T} @)ys (%z, (t)) + 25> Xi, (t)) + el 


so that 7), 7,()&(w) is J’-measurable, 
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Putting §=y7,(*,) (4>T), we conclude that 


{x,ET} X fs, TIE S*. 


Now let X be a process continuous from the right in the 
topological measurable space (E, @, #) subject to conditions 
1.9,4-1.9.B, We write §* for the c~algebra in © x /* genera- 
ted by sets 2x /}(T>s) and set (5.25). Let ¢(t) be any # - 
measurable function satisfying the inequality -<(t)>+, and 
let the points f{ define a canonical sequence of subdivisions 
{At} of the interval /*, We put 


o,(t)=th aif oC. 
For any TC ¢ 


{@. w): X2,, (t) () r} =U {* 2€T] x {9 (2) < Az E 3°. 
The function *:,()() is therefore S°-measurable, As noo, 
@n(4)) ¢(f) and in view of the fact that * is continuous from 
the right, *2,(9>%*»@. By lemma 1.9, *, (y(w)is also §*- 
measurable, Therefore #°— #*, 


5.8. Theorem 5,2, Let X be a strictly Markov process, 
and + a random variable independent of the future and s-past, 
Let ®(w, t)(wE 2, t€/*) be an g*-measurable function such that 
O(w, t)is P, .-summable, Then 


Mz, 2(®(w, *)Jo#}=F rx.) (2.0.2. P,) (5-26) 


where 
F (t. y) = Mz, ,P(w 2). (5.27) 
Proof, Let 
Bl, His St CT. x, @ETy-.0. HET}, (5-28) 
where T>s,I,, -...T,CM and 9, ..., o, are #*-measurable func- 


tions such that 9;(f)>t. If Dw, t)=—x~,, we have 
Ow, = Zs, 7) O Xp, [%. wl + - Xr [He 


1,6.F 
Hence (see ) Ms, i {® (, t)[ of} = 


=X.<7Ps, x {rs ET, ... x, ET yf oA] (a.c,&,, P, a)» 
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where x,=¢;,(:). By theorem 5,1 it follows from this that 
Ms, 2 {8 (o, DM} = Kc rp? He WT +++ Mae Tn). (5.29) 


On the other hand, 
Mey P (0 = Ho, GPC YM OTe +++ On. Tr) (5.30) 


On comparing (5.29) and (5.30), we conclude that relation- 
ship (5.26) is satisfied for ®(v,t)=x%,. Sets B of the 
form (5.28) form a x-system @:, The functions ® for which 
(5,26) is satisfied form an _¥-system (. denotes the class 
of all functions © such that ®(w, <)is Ps,2-summable), By 
lemma 1.2, this system contains all the functions of Y@ 
which are measurable with respect to o(@)=J°, 


Corollary. Let X be a strictly Markov process and let 
the random variable + independent of the future and s-past 
satisfy the inequality *<T. Then for any \’?-measurable 
and P,,2-summable function &, 


Mz. (| of) = M., of (a.c ° Q., P, ae (5.31) 


This may be seen simply by applying theorem 5.2 to the S*- 
measurable function Xie, 7) (f)E(m) . 


Remark, In an important particular case theorems 5,1 and 
5.2 and their corollaries are applicable to any Markov, and 
not merely to strictly Markov, processes, This is the case 
when the functions +, 7, ..., 7, take only a finite or denumer- 
able set of values in Q., For, in accordance with lemma 5,4, 
conditions 5.3.B-5,3.B’ are fulfilled for these functions, 
so that all the proofs contained in sec, 5.5-5.8 remain 
valid, 


5.9. As an application of the general theorems just proved 
we shall mention some properties of the random variables 
5e(1).&, (oF ), t,(G) defined in articles 2-3 of Chapter 3, 


Lemma 5,7. Let X be a strictly Markov process continuous 
from the right in the topological measurable space (EF, @, #) 
and satisfying conditions 1.9.4-1.9.B, Let ¥ be a normal 
system of subsets of (FE, @, #)and §,(%) the instant of first 
departure after 't' of the trajectory of the process from 
the system ¥ (see sec. 3,11), Then the functions §,(F, w) 
G() >t>s) and Xe,(¢, ») (v) are J*-measurable, 
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Proof, Let {I,} be a sequence satisfying conditions 3,.12.A- 
3,12,C and equivalent to ¥. We observe that 


(Fs o)= lim i E+NA, OM Cw@>t>9, 


where A is the set of all non-negative rational numbers and 


-+ ©0, if %,+2()JET,, 
1 for the remaining(s, 2). 


Fn, 2 =| 


The functions /,,,(w. f) are easily seen to be §*-measurable, 
This means that the functions (¢-+r)f,,,(w, 4), and hence also 
the functions &(,w) are S*-measurable, 


Since the mapping of the space (2x, §*) into (E, #) defined 
by the function x,(w) is measurable, we can prove that *:,(, a) 
(w) is measurable simply by showing the measurability of the 
mapping f of the space (Qyx /*,S*)into itself given by the 
expression 


Bw, )=(0, &(F, o)). 


By lemma 5.6, we can show this simply by verifying that 


B'{2QX MES (T>s) 
Stix,Er] XI} EF? (uu >T>s). 


The first of these inclusions follows from the g*-measura- 
bility of the function &(%, w) proved above; the second 
inclusion follows from 


B~" (x, €T) X Mr} = (xn ET (FI <T) = 
= {x, ET X Iz} 1 {& (F) < T}. 
This proves the lemma, 
Theorem 5,3. Let X be a strictly Markov process contin- 
uous from the right in the topological measurable space 
(E. @, @) and satisfying conditions 1.9.A-1.9.B. We put 
No(s. x 4. TY =P a {ts(QEA, x. ET}, (5.32) 


where GE CN @ and +,(G) is the random magnitude defined in 
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sec, 3.13. Then we have for any random magnitude 7 not 
depending on the future and the s-past: 


P, > It, (G)EA, x. (ay€ Tf #5} = IIg¢(%, x. A, DP) 
(A.C. oa P., x): 


(5.33) 


If UECN#, the closure of U being compact and contained 
in G, we have 


P., 2 {*s (G)EA, ¥z, (G) ET eM, w)] =a (5 34) 
= IIg (*s (U), x3, (v’)s A, I’) (a.c 7 2. (uP, x) e 


and 
Ile (s; x A, t)=— f [tce. X at; dy) Ig ¢, Ms A, Y). (5.35) 
Ez 0 


When the process X is continuous, relationships (5,34) and 
(5,35) are satisfied for any open measurable sets UcG, 


Proof, In view of lemma 5,7, the function 
O(t, wo) =7,(%(G ©) Zp [~<,(G, a (w)] 


is S*-measurable, If we apply theorem 5.2 to this, we arrive 
at relationship (5.33). Equation (5.34) follows from lemma 

3.9 and (5.33). In the case of a continuous process, we can 
use lemma 3,10 and relationship (3.54) instead of lemma 3,9, 


3. Stationary Strictly Markov Processes 
5.10. Let X=(x, %, ff; P, =) be a stationary Markov process 


and let 4, be operators satisfying conditions 2,5,A-2.5.C. 
We take any non-negative function +t() and put for any A€d\*: 


A= U (6,4, = (w) = ¢}. (5.36) 


20 


It follows from 2,5,A-2.5,B that operators 6, possess the 
following properties: 


5.10.A, 6,2,=2.,6.(A\ B)=6.4A\N6.B, 


5. (Ua) =Us.4,, 6.14) =118.4, 


(x runs through an arbitrary set of values), 
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5.10.8, 6. 1X}, ET} =F [x, < Yj. 


+h 
Let ¢(w) be an c\“=measurable function, We define the func- 
tion 6.5 by the expression 


bi=—tie), if tw)=—t. (5.37) 


It may readily be seen that properties 2.6,A-2.6.28 and 
2.6.G of operators 6, still hold when t is replaced by =, 
In particular, 


ea (5.38) 


Lemma 5,8, Let X be a stationary Markov process and let 
=(w)(w€ Q,) be an arbitrary \°-measurable function, Then the 
function (uw, 4)=6,(wjis measurable, 


Proof, Let 2 denote the class of ali functions :(m)(wE 2). 
Obviousiy, the class 5¢ of all functions : for which the 
assertion of the lemma is satisfied is an .Y-system, Further- 
more, the characteristic function /z of any set B= {x, €T),..., 
x, €T,} Gp... %2>0; T,....T,€H) belongs to d#’, These sets 
form a =~system generating the c-algebra WN, The statement 
of the lemma therefore follows from lemma 1.2, 


On combining lemma ©.8 and theorem 5,2 and taking 2.5.0 
into account, we arrive at the theorem: 


Theorem 5,4. Let X=(x,, %, off, Ps, 7) be a stationary 
strictly Markov process, and « a random variable independent 


of the future and s—-past. Let é be an wW-measurable function 
such that M, ,6.§ exists, Then 


Ms, 2@8)o#)=My=*)  (a.c. 2. P, ,). (5.39) 
We have for any Bé.\*: 
Ps, 2 (0,8fo%#%)=P,.(B) (ac. 2., Py, 4). (5,40) 


Corollary 1. Let the assumption of thecrem 5.4 be ful- 
filled, Then for any 4A€c#!, BE ds"? 


/ 
Py a(4, 9,B)= f Pz, (B)Py, » (dw). (5.41) 
A 


*)We write Mt instead of Mo,,: and p,(8) instead of Po, z(8). 
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If the function =(w) is W-measurable, the function 7(w) 
oWemeasurable and 7§,. and 9: are P, ,-summable, then 


Ma, » (78:8) = Ms, 2 (1M,.). (5.42) 
Gorollary 2. If vw.) (@€2,,2E(0, 00)) is an arbitrary 
WX #\-measurable function and if (uv. f= bbw. ft), we 


have for any random variable <= independent of the future and 
S—pasts 


Mg, x {D (w, 2) | 02} = F(z, x.) (a.c.2., P, a)» 


where 
F (t, y) = My} (0. f). 

These expressions are deduced from (5.39) by normal use of 
lemma 1,2, 

Theorem 5.4’, Expressions (5.39), (5.40), (5.41) and (5.42) 
likewise hold in the case when BEN and § is (measurable, 
(Measure P, is to be continued into .#0 as described in sec, 
2.2). 

The proof of this theorem is very similar to that of theo- 
rem 2,1. We shall therefore omit the details and just give 
the broad outlines. Let A€ of}, BEeW. We take the measure 
in #: 


p(T’) = Pz (A, x,€T) (TE #) 


and choose B,, B, from a such that 8} © BS B, and P,(B,) = P,, (Be). 
On applying forma (5.41) to 4, B; we get 


P,(A8,B;) = f Po, (Bi) Pz (dw) = f Py (B;) (dy) =P, (B). 
A £ 
This gives us 
P,(A8,B) =P, (Bi). 
On the other hand, 


P, (Bi) = f Po, (B) Pz (de), 
A 
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and Siew is proved for sets A,B, Expressions (5,39), 
(5,40) and (5.42) are deduced from (&.41) with the aid of 
arguments which will by now be familiar, 


5.11. Let §& be random variabies satisfying ccnditions 
3.7 .A-3.7.0 and (3.90). We have 


Giy=§—z (wEQ). (5.43) 


Moreover, for any aE 3 
6. {= a} == oUt, =u, X,=a)}= 
U 
= ios, = a, 6.x, =a} = Ue —-=u, <24=4j= (Xe, =a} 
1“ th 


so that 


Ox. — Ke. ‘ 


(5.44) 


Let f(t, x) (({>0, x€E) be an arbitrary #00) #-measurable 
function, It follows from (5.43)-(5.44) and 2.6.8 that 


Of Go: X= fE.—% Xz). (5.45) 


We now make the assumptions that process X is strictly 
Markov, that « is a random variable independent of the future 
and s-past, and that the function M, ,fG—7z+*:,) is P,.- 
summable, On applying theorem 5.4, (5.45) now gives us 


Mo, 2 (Ff G.— % %_) | of2} = Mp f Bo: X25) (5,46) 
(a 2& r 2, P, 2)#). 


Let G be an admissible set for process XxX, 


*)In order to apply theorem 5.4, we have to verify that 


function /(S,x,,.) (we2o)is off-measurable, In view of 5.7.3, 
Gg>>tt}ex? for any sy y0 . The function %(~)(€%) is 
therefore a random variable independent of the future and 
Q-past for the process (+,,5, #7, Ps2). By lemma 5.2, x, is 
measurable with respect to #{. Since #9 and 2 are 
contained in WW, it follows that % and “% are measurable 
with respect to pWf=\[e,] , whence we may readily deduce the 
eV -measurability of f(b, *:), 
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We introduce the following notation: 


a(x, T= f e#G@P, (dw) (> 0), 
Ti@ést 


Te f (x) = Mze-* fxg l= [xe (x, dy) FO), 
E 
TQq(x, T)= 74 (x, PT) =P, {x1 ET}, 
Mg f(x) = 1S f(2) = Me f1%, l= [Fe 9) 10). 


B 
M(x) = M_5(G). 


Theorem 5,5. Let U,G be admissible sets for the station- 
ary strictly Markov process X, where UGG. Then the 
following relationships hold: 


ef (x) = Mh f (x) (5.47) 


(if e-¥(@f[x:q@]is P,-summable), 


ne (x, T)= f xy (x, dy) 56(y, I), 
Ez 


Mag(x) =m, (x) + Mm, [x; ay! = 


(5.48) 


= My (x) + fg Ro (x, dy) (5.49) 
£ 


(if me(x)<00). 


Proof, We put in formla (5.46): s=0, f(t, x)= e-* f(x), 


§=§,(G). t= §(U). On taking into account the fact that &= 
g(G)by virtue of (3.54), we have 


: a 
M,, {e748 @)-§(ON F [x, ‘@) |] 42 pj =Me FX (yy) a,c 2. (7,P2)- 


Since the function ,i:() is Mz (7) -measurable, it now follows 
that 


Mz {e-*5@ f1x¢ aq] JoMe } = e-* CITY f [xe] 
(ac. Qe; P.,) 


On taking the mathematical 


expectation of both sides, we get 
(5.47). 


With f(x)=y,(x), relationship (5.47) becomes (5.48). 


Relationship (5.49) follows from the series of equations: 


STRICTLY MARKOV PROCESSES 123 


M,§ (G)— Mt (U) = f [(G)—§ (U)1P,, (de) = 
®&(T) 
= [61% G) Pz (do)= MM, &(G). 
F &(T) 
&(U) 


Remark, Let X be a stationary strictly Markov process 
continuous from the right in the topological measurable 
space (EF, @. #)and satisfying condition 1.9.B, and let U, G 
be open measurable sets in (£, @, #), the closure of U being 
compact and contained in G, If we replace §,(G) by 1,(G) in 
the definition of functions Té/f(x), xs(x.T), and mg (x), rela- 
tionships (5.47)-(5.49) remain valid, (This follows from 
lemmas 3,7 and 3,9.) 


5.12. We shall now indicate briefly how our definitions and 
results are modified if the stationary process is understood 
as the system (x,, C, o#, P,, 4) subject to conditions 2,8,A- 
2.8.H, a 


We shall describe the process X= (x, C, of, P,, 6,) as measur- 
able if the mapping of the space (Q;X J, of, #,) into(E, #) 
defined by the function x,,(w)is measurable for any ¢, We 
shall describe the non-negative function <(w)(w€2) as a ran- 
dom variable independent of the future if t(w) <C(w) and 
{tct<t}€o#, for any +, If ACQ.={[r<f} and {A,7-<t<6} 
CoM, for any t, we shall put A€o#,. Ali the lemmas of 
article 1 are easily carried over to the present case, In 
particular, we can assert (see the remark at the end of .sec, 
5,2) that, for any measurable stationary process, the transi- 
tion function P(/, x, T) is always a measurable functimof {tx}. 


The process X=(x,,¢, of, P,. 6,) will be described as strictly 
Markov if we have for any random variable +t independent of 
the future, for any non-negative o#.-measurable function 3(w) 
(»€ 2.) and for any TE#: 

Pp {x.3€P]o#,} =P, x, T) (a.c.2.P,). (5.50) 
It follows from this that, for any 8¢€.jN: 
P,{8,B]o%.}=P,,(B) (8.0. 2, Pz) (5.51) 


and for any df'-measurable function § such that g& is Pz- 
summable: 
M, {9,6Jo#,} =M,& (a,c, &,, P,). (5.52) 
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i 


We remerk in conclusion that all the expressions deduced 
in article 3 for stationary strictly Markov processes hold 
for all stationary Markov processes if the random variable 
* takes only a finite or denumerable set of values, 


4, Weakening the Form of the Condition for Processes 
Continuous from the Right to be Strictly Markov 


5.18. Throughout the remainder of Chapter 5 we shall con- 


sider Markov processes continuous from the right in the topo- 
logical measurable space (F, @, #) and satisfying conditions 


1,9.A-1,.9,B. We can weaken for these processes the formula~ 
tion of the strictly Markov property 5.3.B by limiting the 
classes of random magnitudes +t and y appearing in the state- 
ment; also, we can conveniently introduce the random varia- 
bles f(x, instead of the events {x,€I}. We agree to write 
C for the class of all bounded contimious measurable func- 
tions f(x) in the space (F, @. #). 


Lemma 5,9. The Markov process continuous from the right 
X=(x, 0, At, P;.o) is measurable, 


To prove this, we simply apply lemma 1.10 to the interval 
A==[s, tf], the measurable space (&,, off), the topological 
measurable space(E, @, #@)and the mapping x, Ww). 


Theorem 5,6. Let X=(x, 0 of, P, .) be a Markov process 
continuous from the right satisfying condition 5.3.4 and the 
following condition: 


5.153,A. Whatever the x€E, 0¢s<t, fEC » the equation 
holds for any random variable «+<f independent of the future 
and s~past: 


Ms, o{f(e)1oM) =M.,o,f(%) @.c.2,P.2) (5.53) 
Then X is a strictly Markov process, 


Proof. The process is measurable by lemma 5.9, Let FEC, 
let t be any random variable independent of the future and 
S-past and 7 an of;-measurable function satisfying the 
inequality %2>*, let the points #{ define a canonical 
sequence of subdivisions {Ax} of the interval [s, oo). We 


put 2 w= t(o), iff't(w)<Z, 
t, if t() > ¢. 
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In view of condition 5.13.4, we have for any ick: 


Ms, 2 { F(x ,n\ | ofa} =F, Xm A] @.0.2.0, Ps, 2) (5.54) 


4 
where 
F (a, y. 0) = My, y f(y): (5.55) 
Let A€of#?. Then the w-set 
Atn={A, EAP, 1 € AP] 


belongs to %n . For obviously, Aw€oM#? andt?(w)—- (w) for 
w€AR,. Theréfore 


(Ate Et <o} = (Ah, t<t< 6} CoM 


Since A%,€ c#in, it follows from (5.54) that 


2 


J F( 2) P,, 2 (do) = fF(st. Xm th) Pe 2 (do). = (556) 
Atk Atk 
We put %n(o)= 4 if n(w)€AZ, Obviously, %n(o) =f, t(w) = +f (w) 


for wCA%, . Relationship (5.56) can therefore be rewritten 
as 


A” 


[f_,) Ps, o(@o)—= [FO x5 9) Ps, odo). 
Atk tk 


On summing over all pairs of values i<k, we get 
J F Gag) Ps, 2 (do) = f FO: x In) Ps, 2 (do). (5.57) 
A A 


We now observe that, since the process is continuous from 
the right whilst f(x) is continuous and bounded, the function 
of (5,55) must be continuous from the right with respect tov. 

On the other hand, 7, |7 as 7c so that 

Ff (%q,) > F (%)- 
On passing to the limit in (5.57) as noo, we get 
SFG) Po, odo) = fF (x01) Poo (de). (5.58) 
A 4 
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Let £ denote the class of all bounded #-measurable func- 
tions and let f€ 4’ if relationships (5.55) and (5.58) are 
satisfied for f. It follows from lemma 1,8 that H2> 7. 
Relationships (5.55) and (5.58) therefore hold for every 
function f€.%, If we put, in particular, f=y7,, we get 
equation (5.8). 


5.14, Theorem 5.7. Let X=(x;, 6 of*, P, 2) be a Markov pro- 
cess continuous from the right satisfying condition 5.3.A 
and the following condition: 


5.14.4, Whatever the x€F. sp 0, h>O0. fEC and the random 
variable < independent of the future and s-past, the equation 
holds: 


Ms, olf“ loMi}=F (x t+h)  (a.c.2,P, ,) (5-59) 


where 
P(u, y. 0) = My, yf (Xp). (5.60) 
Then the process X is strictly Markov. 


Proof, It is sufficient to show that condition 5.13.4 
follows from 5.14.A, let fCC and let + be a random variable 
independent of the future and s-past satisfying the inequality 
t<?f . We take the canonical sequence of subdivisions {Af}. 
of the interval [s, t] defined by the points 


=stt¢—s) (@=1,2,....m n=1,2,...). 
We put 
hy t —iRy, 
8,(u)y==u-+hy, if uC dj. 
By (5.59), 


Me, o {f(x 2np\[oMs} =F (sx tht) (2.0.2, Py, 2) (5.61) 


Let A€of#;. Then Ap={A,t€Az} CoM: , and by (5,61): 


J f(%+np) Po, 2Gde)= JPG tot HADPy, o(do). (6 65) 


n 
Ai 


But 8,(7)=t+aAe for w€ AZ. On substituting this value in 
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(5.62) and summing over k, we get 
f flea olPs,2(do)= ff Fit, x. B,C) Ps, 2(d). (5,63) 
A A 


We now notice that 8,(u)|¢ as n-co, Om passing to the 
limit in (5.63), we get 


f(x) Ps,2(do)= [ F(z, x,, DP, , (do). 
f J AP, _ (de) (5.64) 


It follows from property 5.3.A that for any ? Fuu,y,t)is a 
measurable function of{u, y}so that F(. «+, O=M, 2 f(x, is an 
ofi -measurable function of w, Relationship (5.64) is there- 
fore equivalent to (5.53). 


Remark, Let Q be an arbitrary family of monotcnic trans- 
formations of the interval [0, ~) into itself, satisfying the 
following conditions, 


5.14.0. y(f)ptfor any 7€Q,¢>0. 


5.14.6. The constant K exists, such that, for any 37¢€Q, 


s, t,>0 5 
lp @)—¢(s)i << Kl é—s]. 


5.14.7. For any 0<¢s<t there can be found a veg such 
that ¢(s)=¢. 


Theorem 5,7 remains completely valid, and no substantial 
changes are needed in the proof if we replace condition 
5.14.a by the following more general condition: 


5.14.a°, For every ¢€Q: 


Me, x {f[%pq@ lbo#t} = F(t Xe, (2) (8.042, Po, 2) 


(147 have the same meanings as in the statement of condi- 
tion 5.14.2). 


Condition 5.14.a° reduces to 5.14.4 when the family Q con- 
sists of the functions 9(t)=/+h (h being any non-negative 
constant), 


It may also be noticed that condition 5.14.2” is equivalent 
to the requirement that condition 5.3.B be satisfied for the 
random variables 1=¢(*) (¥€Q). 
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5. Strictly Markov Subprocesses 


6.15. Is the strictly Markov property preserved when pro- 
cesses are transformed as described in Article 3 of Chapter 2 
and in Chapter 3? It is easily verified that, if a strictly 
Markov process X is subordinate to a measurable Markov pro- 
cess X (see sec, 2,11), X is also strictly Markov, The 
converse does not hold: a non-strictly Markov process will 
be constructed in 5,22 by widening the basic c-algebras of a 
strictly Markov process. We are still less entitled to expect 
the strictly Markov property from arbitrary subprocesses of a 
strictly Markov process, It will be shown, however, that 
given very broad assumptions, the class of equivalent sub- 
processes of a strictly Markov process contains a strictly 
Markov process (this being the canonical subprocess described 


in sec, 3.3). 


The fundamental result of the present article is stated in 
theorem 5.8, We shall first prove some lemmas, 


5.16. Lemma 5,10, If the strictly Markov process X satis- 
fies condition 5.3.A, this condition is satisfied by every 
subprocess of X subject to requirement 3.5.8. 


Proof. Let TES, 0<scu; s<t, xC€E. We put 


M, Xp (x,) if tc, 


Fis, x, j= 
My uX%p(%,), if t>a. 
Let A be any positive number, We select points r,—0, 
Ty, ...5 Fy,o=u Such that 


%2— = 93— hh = “oc ly, — Ty_1 =A, O<n—mH<h. 
We consider the function F(s, x, 4) in the set I7) Xx EX 
t+1 


[ee sy. Tt is continuous from the right in ¢ (by virtue of 
3.5.8) and #3 =X #-measurable with respect to s and x (by 


lemma 5,3), It is consequently (see lemma 1,10) 9% xX xX 
"! 1 


B ine -measurable on the product set (whose elements are the 
triples){s, x,t}, We conclude from this, on the basis of 
lemma 1,3, that F(s, x, sth) is a By _X% #-measurable func- 


: + 
tion of s and x on the set if XE@=0, Ij Qin chia RRO) s 


*\We put i;=[s,t) and write #; for the c-algebra in the space 


f; generated by all the intervals contained in 7, 
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This function is independent of s on the set Dame ao and is 


clearly also measurable (see lemma 2,2), Hence F(s, x, s+) 
is a#.X #-measurable function throughout the set Ix B-% 
By lemma 3.1, 


P(s, xu, T) =F (s..X, $) = lim F (Gs, x, +h) (cE R, x EE). 
yo 


Hence P (s, x; u, This a xX #-measurable function of s and 
x, and the subprocess satisfies condition 5.3,A, 


Let 3X be a subprocess of the Markov process X, Let «t be 
a random variable independent of the future and s-past for 
process X, We shall show that <= min(c, ) is a random vari- 
able independent of the future and s-past for the subprocess 
xX. In fact, by 3.1.0’, 


it<tcte=[r<tct t <P} eof, 


so that condition 5,1.B is satisfied, Condition 5,1.A4 is 
obviously also satisfied, We observe that AQ~ €of#* ifAC oh. 
For we have for any t>0: 


{A, Qe, tt PH lA tt > TSF} Cf. 


Lemma 5,11, Let X be the canonical subprocess of the 
Markov process X (see sec, 3,3). Then an arbitrary random 
variable < independent of the future and S-past for the sub- 
process X is equal to min (c, ¢) , where t is a random varia- 
ble independent of the future and s-past for process X, Any 
event A€ of! may be written in the form 4=Af @-., where 
AE ch, 


Proof, Conditions 3.3,4-3.3.E define 8,0 x, o#% and off® 
for the canonical subprocess, We write C, for the aggregate 
of AE[0, co) such that (w, \) €Ck~, and ©. for the set of all 
w€2 for which C, is non-empty. We put 


| sup t(w, 4) for wEQ., 


aa (5.65) 


T(w) = 
max [s, € (»)] for w EQ. 


We have for every t>s 3 
{p< t< 2} Cok} 


so that 
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[T<t<t|=BX (t, oo], (5.66) 


where BEof;., On comparing this equation with the obvious 
inclusion 


we get 


On the other hand, it is clear from (5.66) that, if mE B', 
we have c(w 1) <t <f(wo for all} >t. Hence it follows, in 
view of (5,65) and 3.3.B, that <(m)<t<2() . Thus 

fc<t<l} = BE of. (5.67) 
We have from (5.66) and (5.67), for any t>s , 
[TotcPleke ct tctjalrcrc®}. (5.68) 


It follows from this that 


and 
cv 


= min(c, 6). 


It is evident from (5.65) that « satisfies condition 
5.1.4. In view of (5.67), « satisfies condition 5.1.5. 


The function « is thus a rendom variable independent of 
the future and s-past. 


Moreover, if A€o#%, we have for any t>s: 


so that 


{4,7 <t<l}=DXF, ool, (5.69) 
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where. D€ of? . From (5.68) and (5.69): 
[A,ct<ct, ton a{D, tod). (5.70) 


We put wéA-if (, €A for all sufficiently large values 
of 4, It is clear from (5,70) that, for »€D,(w.4) € (4, +<# 


DS{A,7<t<%}. 


On the other hand, if m€{A.t<t<C}, we have (% AE{A, < 
<t<t, t<d}={D, ¢<)} for all sufficiently large i, so that 
w€D. Therefore 


[A, =< t<t} = DE of}. (5.71) 


This shows that A€o#2. It is clear from (5.69) and (5.71) 
that 


so that 


Lemma 5,12, If a= {aj} is a multiplicative functional and 
* a random variable independent of the future and s-past for 
the Markov process X, the function 2} is of?-measurable*), 
If X=(x,, f, off, B,,,) is the canonical subprocess of X cor- 
responding to the functional a, we have 


P, {- <  [o4 } =o (aC. 2. Ps, 2). (5.72) 


Proof, let s<t. The function ai,(w)(ZEJ;,4€ 2, is con- 
tinuous from the right in 4 for any w€Q, andis of#- 
measurable in w for any 4€f. By lemma 1.10, this function 
is 1X of#j -measurable on the product-set (u,w). On the 


*)The o-algebra Ai is constructed in accordance with the :- 


algebra e#* just as #! is constructed in accordance with -4? 
(see sec, 2.2), It is easily seen that Acc#* when and only 
when {4,t<1<t}ews for any ¢, 
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other hand, the restriction of the function 7(“) to the set 
ea (0) rea (w)! defines an of#;-measurable napping of this set 
into the segment /? , The restriction of a, .)(w) to the set 
f=(w) <é<%(w)!is thus of] -measurable, which proves the first 
assertion of the lemma, 


Further, let A€o#*, We put B={A,7<f}. Then in the 
notation of sec, 3,63 


p =| (*(), co] for w€ ANg,, 
a 0 for w€AN2@.. 
Therefore 
WB= AR = Bets WES) 
and 


P a (B) = M, ap ze M,, wtahar 


which is equivalent to relationship (5.72). 


Lemma 5,13. If «= (ai) is a multiplicative functional 
satisfying the condition 


5.16.4. net (©) =ai(w) for all 0 s<t<Cw), 


then the function 
flu, o)=af() (jE [s, ft], o€ 2) 
is 3*-measurable, 


Proof. Let the points 1% define a canonical sequence of 
subdivisions {AZ} of the interval [s, 4] (see sec, 1.9). We 
put 


Pn(“)= th if uw Az. 
By virtue of condition 5.16.4, 
nim, a7 (a) = at (a) = f (a, 0). 


it is readily seen t ab 
ain) (wy) me 2 Xam ey a E(w), 
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It follows from this, in view of lemma 5.6, that the function 
as” (w) (w) is §*-measurable, The limit function f(z, w) is there- 
fore also SJ*-measurable, 


5.17 Theorem 5,8. Let X be a strictly Markov process con- 
tinuous from the right in the topological measurable space 
(E, @, #) and let a= {a§\. be a multiplicative functional for 
process X satisfying condition 5,16,A. Then the canonical 
subprocess xX of xX corresponding to functional « is a 


strictly Markov process, 


Proof, Since the subprocess X is continuous from the 
right and, in view of lemma 5,10, satisfies condition 5.3.4, 
we only need to show that it also satisfies condition 5.13.4 
(see theorem 5.6). let fEC, <<¢ be a random variable inde- 
pendent of the future and s-past for X,, and A4€o#%. We 
consider the corresponding random variable + and event A 
constructed in lemma 5,11, It is clear from (5,65) that 
c<t. We have by lemma 3,1: 


M Yof D=M, Xf te, =M, Fee (5.73) 


By 3.5.B, a= aa; and we obtain on taking lemma 5,12 and 
1.6.H into account: 


My, oXal (4) of = My, (Xs, [fF (,) af of]}. (5.74) 


It follows from lemma 5,13 that the function f(x,) ar (aE fs, t], 
w€2,) is §*-measurable, and by theorem 5,2: 


Ms, « (f (x,) «| Ms} = F(t. x,) (8.0.2. Pec) (5.75) 
where 
F (a, ¥) = Mu, of (#) af = Mu, of (). (5.76) 
We conclude from (5.73), (5.74) and (5.75) that 
M, axxf (&)=M, .x4a8F ( x). CHEe: 
On the other hand, in view of lemma 5,12: 


M, aX F (*. x~) = M, oe (* x ) kes s = (5 78) 
=M, wAxaF @ «BP, [C> to ]}=M, wxaF ( x)az~* 
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Comparison of (5.76), (5.77) and (5.78) gives us (5.53), 


Tre reader will easily prove the following variant of 
theorem 5.8 for stationary processes in the sense of sec, 
2,8. 


Theorem 5,8° . Let X=(x, 3, 0%, P, 4) be a stationary 
strictly Markov process (see sec, 5,12) and let a bea 
multiplicative functional of X for which condition 3,22,8 
is satisfied in the following strengthened form: a,8,a,—-a,., 
for all s,f>0 and w€&,,,, Then the canonical subprocess 
corresponding to 2 is strictly Markov, 


6, Criteria for a Process to be Strictly Markov 


5.18. Theorem 5,9, A sufficient condition for a Markov pro- 
cess continuous from the right to be strictly Markov is that 
its transition function satisfy the following condition: 


5.18.4, Whatever the f€C , the function 
F(u, yy= [ Pw, yi t, dz) f@) (5.79) ° 
E 


possesses the following continuity property: 


lim F(a, y) = ‘. 
ee eT =) (5.80) 


uYS 


Proof, We write .— for the class of all #-measurable 
bounded functions in £ and 3 for the class of all functions 
fEL such that 


Fi, y= f P@ y; t, dz) f(z) 
E 


is a #,x)X @-measurable function of u and y, We note first 
of all that, by lemma 1.7,F(u, y) is a @-measurable function 
of y for any S€.f and any u>0. If fE€C , condition (5.80) 
is satisfied in addition, and by lemma 1,10, F(u, y) is a 

#10, ; X #-measurable function of wu and y. Thus w2Cc. But 
ot is obviously an -systen, and by lemma 1.8, 57> 7. We 
see on putting f=y,T'€ #) that the function P (a, yn t Yr) is 
measurable with respect to u, y, Condition 5.3.4 is thus 
fulfilled, In view of theorem 5,6, it remains to verify that 
5.13.4 is also fulfilled, Let f€C , and let t be a random 
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variable independent of the future and s-past such that 
c<t., Let the points ¢; define a canonical sequence of sub- 
divisions {At} of the segment [s, 4] . We put 


tr (w) =f, if (w)E Az. 


The random variable <, takes only a finite number of values, 
Expression (5.31) is thus applicable to it, in accordance with 
the remark at the end of sec, 5.8, and 


Ms, {f(e)fodi,} =F Gu te,) (8204 2:,5 Ps, 2) 


2) 


where F(u, y) is given by (5.79). The restrictions ofz, to 
Q are clearly o#2-measurable, and by lemma 5.1, {4,7 <71€ 
of? for any ACcM!. We therefore have for all n and any 
nm 

AC of: : 

M, chal. <f(%)=M aohalrycd (Gu *.)s (5.81) 
We observe that t:1* for n-»oco. Obviously, 4) <: 7h <: 
=%o. Using condition 518.A and the fact that the process 


is continuous from the right,we have Pulte xe DoF x.) » On 
passing to the limit in (5.81), we get 


M, hal (~,) oa M,, gear (7 x,). 


Since the function F(x, x.) is of, measurable, this relation- 
ship is equivalent to (5.53). 


Remark, As is clear from the proof, the assertion of 
theorem 5,9 still holds if the transition function satisfies 
condition 5.3.4 and the following weakened variant of condi- 
tion 5,18.A. 


5.18,4°, For every f€C and w2 , the function 
O(w, 4) =F (u, x,) = [Pu x,; t, az) f (2) 
EB 
is continuous from the right in regard to uz. 
5.19. In the stationary case it is advantageous to modify 
slightly the continuity condition 5.18.4 and to replace it by 
the following: 


5.19.4, Whatever the fE€C, the function 
F(y)= f Ph, y, dz) f(z) (5,82) 
E 
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is continuous with respect to y for any A, 


Condition 5.19.4 was first discussed by Feller, and we 
shall refer to stationary Markov processes satisfying this 
condition as Feller processes, In general, if condition 
5.19.4 is fulfilled for a stationary transition function, we 
shall speak of this as a Feller transition function, 


Theorem 5,10, A Feller process continuous from the right 
is strictly Markov. 


Proof, In view of lemma 5,9 and the remark on lemma 5,3, 
condition 5.3.4 is satisfied, By theorem 5.7, we only need 
to prove that condition 5.14.4 is also fulfilled. The proof 
follows the same lines as that of theorem 5.9. 


Let fE€C and let t be a random variable independent of the 
future and s-past. Let the points 4 specify a canonical 
sequence of subdivisions {a7} of the interval [s. oo), 


We put =,(o)= if t(w)=A?, In accordance with the remark 
at the end of sec, 5,12, we are justified in applying expres- 
sion (5.39) tc t,, so that 

Me, x (f (x:,+n) lo: } = Ma, f (x,) =F Cm) (a,c. Q. Ps, x) 


where F(y) is given by formla (5,82), If A€o#, then 
AG of: , and 


[1% 42) Ps,2 (do) = fF (x.,) Py, 2 (do). (5.83) 
aA A 


As n-ocot,|t, On passing to the limit in (5.83) and 
using condition 5.19.4 and the fact that the process is con- 
tinuous from the right, we get 


I (es ) Ps, x (dw) = F (x, P,, - (dw), 
J ; : (x,) Ps, 2 (dw) (5.84) 


whence (5,59) follows, 


Remark, The above proof of theorem 5.10 remains unchanged 
if we renlace the Feller condition 5.19.4 by the weaker con- 
ditions 


5.19.4°. For any f€C, w€Q, the function 
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F(x) = [ Ph, xp dz) f (2) 
E 


is continuous from the right with respect to ?¢, 


5.20. We shall say in future that the Markov process Y= 


(x, © of: Ps, 2) is continuous from the right at the point x-if 
(,== x} S{ tim x,—=x}for any t>0. We shall say that the 
aU 


stationary Markov process X is of Feller's type at the 
point x if, for any f€C , the function F(y) given by (5.82) 
is continuous at the point x, A word for word repetition of 


the arguments used in proving theorem 5,10 leads to the 
following variant of this theoren, 


Theorem 5,10’. If a stationary Markov process is contin- 
uous from the right and of Feller's type at every point x of 
some subset G of the phase space £, condition 5.3.5 for 
being strictly Markov is satisfied for any random variable + 
such that «,€G for all w€2,. Theorem 5,9 can be similarly 
localized, 


§.21. The conditions imposed on the process in theorems &,9 
and 5,10 remain valid for any widening of the basic s~alge- 
bras, Hence if these conditions are satisfied, not only the 
process X, but any process cbtained from X by a widening 
of the basic sc-algebras, is strictly Markov, and the same can 
be said of any process subordinate to X (see sec, 5,15). 


We put A€oMi,) if ACG, and {A.C >v}CoM, for any vu>?t. 
Obviously, of#,, is a c-algebra in the space Q,. 


Theorem 5,11, Let X=(x, 0, oft, Ps, 2) be a Markov process 
continuous from the right in the topological measurable 
space (E, @, #). Let X satisfy condition 5,18.A and be a 
stationary Feller process, Then X’=(x, %. ofiio0 Py, z)is also a 
Markov, and indeed a strictly Markov, process, 


Proof, In view of the remark just made, we only need to 
prove that xX’ is a Markov process, The system xX’ clearly 
satisfies conditions 2,1.4-2.15.E, so that it only remains 
to prove condition 2,1.F’, 


We start by supposing that X satisfies condition 5,18.A, 
Let f€C,0<¢s<t<cu and let vE¢, uw]. By (2.5), 


Ms, 2 f(x, ) | oM3} =F (a x) (Aa Qe Pr, 2) 
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Es 
Sos 
6) 


wnere 
F (wv, v) = Me, yf (%y)- 
If A€oM.0, then [4,2 >v}€of#,, and 
[ f@)P.2@o)= f Fle x) Po, 2 (de). 
{4, > v} {A,% > vy 


On passing to the limit here as v|t and taking into 
account 5,18.A and the facts that x, is continuous from the 
right and f continuous, we find for any f<4a: 


[fd Pex (do) = f FG ©) Pao (do). (5.85) 
A A 


On letting «1+, we find that this equation is also valid 
for u=/ , Moreover, it may easily be deduced, on the basis 
cf lemma 1.8, from the fact that (5.85) is satisfied for all 
fec , that these relationships are satisfied for every ¢- 
measurable bounded function f (ef, the end of the proof of 
theorem 5,6). 


Putting f=yr , we get 
Py, ofA: x,ED= fre Xe; “, I) Ps, x (dw), 
A 


which is what we wanted to prove, 


Now let X be stationary and satisfy condition 5.19.4, Let 
fEC UKsgt<u , In accordance with (2.5), 2.6.F and 
2.5.B, we have for every <=>0 3: 


M.,, x (f (Xu+s) | A,..| =F (Xt+2) (a of oQuie Ps, a) 
where 
F(y)= Mex, ut (4y1.) = Myf (%u-2)- 


Let A€of#i.o. Then {4,0>t+¢) € off,, and 


fF @ui)P oo (do) = fF (e144) Pe, 2 (do). (5.86) 
A A 


On letting :|0 , we have 
J £0) Po,2(du)= f F(x) P., 2 (do). (5.87) 
A A 
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Just as in the first half of the proof, we can show that 
(5.87) is satisfied for all #-measurable bounded functions 
f. We have with f=yr: 


P,2(A, x,€N= f Pit, x5 uw, T)Py,2 (de), 
aA 


and we have now established that condition 2.1.F’° is satis- 
fied, 


Corollary. (The law of zero or unity.) If the assumptions 
of theorem 5,11 hold, P,,~(A) is either zero or unity for 


any A€ offi ond. 
For, in accordance with theorem 2,1° and 2,1.E, we have 
Pe, c(A]oMs0) = Ps, 2, (A)=Ps,2(A) (2.0.25, Py, 2). 
On the other hand, in view of 1.6.A, 
Ps, x (Af oAi,0) = x4 () (2.C.2,, Ps, 2). 
Our assertion follows on combining these two equations, 


Remark, The necessary and sufficient condition for a func- 
tion <(w) satisfying condition 5,1,A to be a random variable 
independent of the future and s-past for the process X = (x5 2; 
oki... Ps2) is that [w t(o)<t<l(w)}€o4#j] for any t>s. 


It is also necessary and sufficient for this that, for any 
e>0 the function t-+e be a random variable independent 
of the future and s-past for the process X=(x,, %, Hi Ps, ae 
(We leave the proof of these statements to the reader), It 
seems natural to describe such functions « as random variables 
independent of the distant future and s-past (for process X), 
If xX satisfies the conditions of theorem 5,11, we can apply 
to variables independent of the distant future and s-—past all 
the formmlae proved in Chapter 5 for random variables 
independent of the future and s-past, 


5.22. In order to apply the criteria obtained in the present 
article for a process to be strictly Markov, we need suitable 
tests for continuity of the process from the right. These 
tests will be deduced in the next chapter. By using these 
in conjunction with theorems5.,9-5.10, we shall construct in 
Chapter 6 a number of important examples of strictly Markov 
processes (see articles 4 and 7). 
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One instructive example may be considered here, however*), 


We suppose that a particle can either move uniformly to the 
right on the half-line [0. co) or remain at rest at the point 
0. Further, if it is situated at the point 0 at some 
instant ¢, it remains there until the instant ‘+A witha 
probability of e-* independently of the time which it has 
already spent at 0. We have here a non-cut-off stationary 
Markov process X=(x,, oN}, Ps.) With the transition function 


Ps,23t,.D=P(t—s, «. H= 
yr({—s+sx), if x>0, 
: (5.88) 


f e-@-#-y0(u) du + yp Oe", if x«=0. 
0 


It is clear from this that, for x«>0, 
F (x)= f Pit, x, dy) f(W=f(x+4 (5.89) 
0 


so that the process is of Feller's type for all points x«>0. 


Let « be any random variable independent of the future and 
s-past, As in the proof of theorem 5.10, we construct ran- 
dom variables 7,11 satisfying condition (5.83). To prove 
that X is a strictly Markov process, we only need to show 
that (5.83) becomes (5.84) as n-+oo (assuming that f€C). 
By (5.89), we can do this simply by proving that 


Paa{t,=0, x, >0 for n=1, 2,...}=0. (5.90) 


We put 
D={w:%,= Q}. 


Suppose that w,, o,¢€D and <t(#)=—t<t(w,) . We have 


C= {D, =f} = it, X,= OE NY 
and by lemma 1,5, 
Xe (w) =f (xe, ..., Xpase. ), (5.91) 


*We leave it to the reader to give a strict statement of 


this example, i.e. to construct the space ©, the function 
x;(*) and the measures Pz, , 
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where f(x,,.... %,--.-)is a @”°-measurable function in the 
space E~ and #,,...,¢,...€[s, t]. Evidently <+,(0,)= +. 
(o,)=0 for all wE[s, t], and yc(“,)=xe(,) by virtue of (5.91). 
But this contradicts the fact that w,€C, »™€C , The contra- 
diction proves that the function 1t() takes only one value in 
set D, If this value is equal to ¢, we have 


D = {x,=0} = (x,=—0, += f}. 
Hence, for any §g>0 ; 
{x,=0, x, >0 for n=1, 2,...)} (x,=0, X43 > O}. 


But in view of (2.15) and (5.88), 


Po, x {X= 0, X43 > 0) = P(s, x; t, 0, t+4, (0,00))= 
= P(s, x; t, 0) P(t, 0;¢-+38, (0, 00)) < P(t, 0; # +34, (0, 20)) = (5.92) 


) 
_ f e~C-du >» 0 
0 


as $810 , which proves (5,90). 


We notice that X’=(x, Wiso, Ps,2) is also a Markov process, 
For we can deduce relationship (5.85) precisely as in the 
proof of theorem 5.11. In view of (5,88) and (5.92), (5.85) 
becomes (5,86) as e]0 . At the same time, x’ is not a 
strictly Markov process, For the instant +=§, of first 
departure of the trajectory from the set {9} is a random 
variable independent of the future for xX’, We put A=&,, 
n=t+1, P={1}. The left-hand side of equation (5.8) is now 
equal to unity, whilst the right-hand side is equal to zero, 


CHAPTER 6 


CONDITIONS FOR BOUNDEDNESS AND 
CONTINUITY OF A MARKOV PROCESS 


1. Introduction 


6.1. The present chapter aims at deducing the conditions in 
which the processes corresponding to a given transition func- 
tion include at least one of the trajectories which possess 
definite properties of boundedness or continuity. 


The fundamental results will be obtained on the assumption 
that the initial transition function P(s, x; ¢, T) is normal, 
i.e, satisfies the condition P(s, x; s, E)=1 for all s>0, 
xCE, The processes corresponding to a normal transition 
function will be described as normal, 


We shall take as our basis the following general theorem, 
Theorem 6,1, Let ¥=(x,, 6 off, Ps,2) be a normal Markov 


process in the measurable space-(F, #) and let YoO&sr *) ; 
We assume the existence of the non-negative function 


GOS. Tye Xie wetn Fe ho kiss) 
(AE [0,+-s0}; tf, -..5 tas». E10, ++ 90); 
Xr vees Xp, ... EE), 
which is a #.X #”-measurable function of A, 1, ..., X_.... 
and which satisfies the following conditions for any denumer- 
able everywhere dense subset {t,, ft, ..., tg ...} of interval 
[s, oo)j 3 
6.1.4. g (A; tis X1> eee, tn. Xn) ee 
= 905 by Xp 20s tye Hf, 2), 


if x,=-x; for all indices i for which 4<) , 


*\The space 2, is defined in sec, 2,11, 
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6,1.B. If -4€l[s,+ cof and Gt. x1, 0... ty ey... )=O , 
definite function % can be found in the interval [0, . oot 
belonging to £ such that 9(¢)—=*i for all 4,€[0, ») *). 


6.1.0. Mg, 2g (3 bys Xt) oes bes Xty? ii) = 05"): 


Then there exists a Markov process equivalent to X such 
that all its trajectories belong to £, 


Proof, We write — for the set got from £-.by adding the 


~ 


extra point a, and @ for the s-algebra composed from all 
sets of the form 8B, BU {a} (BEF). 


We put % (w) = + 09, 
x; a: xt (w), ‘= f< 6 (), 
a, if t>£(), 


Ps,2(A)=Ps,2(A) (EE, AEA). 

We use .% to denote the c-algebra in the space 2 genera- 
ted by the sets {w:x,(w)ET} (wels. 4) TEM). It still 
remains to define P,,(A). It is easily shown that every 
set A€o* either does not intersect with @=(t<s}, or 
else contains ©*. We put 
(oO, af And=@, 


P,, A)-=' ne 
ke [1, if ADO xx), 


*)To fulfil condition,6.1.B with A=s we simply require .¥ 
to contain any function with '(0,s) as its domain of definition. 
**)The function 97(5 ty *4,,--+) tn: *t_,---)cam be regarded as 
uniquely defined for all wee, For, although the values 
of the xt, are not defined for 4¢4,>¢, the expression 9 (5 4, 
Xy>+++> tn, Xtz,+--)does not depend on ~:, for t>¢ by virtue of 
6,1,4. We can therefore assign arbitrary values to x,, for 
tn >t Without their choice being reflected in the value of 

QC; ty, Xp 5-2) Ens Heys oo 
***)This definition is incorrect if 98s.g, since for every 
Aco*® we now have simultaneously A>&* and «apa*=g. But we 
can suppose without loss of generality that ())=0 for some 
wé€2, %$So that o? is non-empty for any s. For, if there is 
no element ») with this property in the space og, we can 
enlarge 2 with the single point % and put C()=0, P, .({%}) 
=0 (s>0,x€£), whilst retaining the previous values for the 
symbols «;() and .#%, It is clear that, with this, a Markov 
process is obtained which is equivalent to the initial pro- 
cess X and which satisfies, along with X, the conditions of 
theorem 6,1, 
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ad ~! 


We have for any 0<s<t, x€E, TER? 


Pis, x; f, T) =P, . ix,ET} = 
Pi(s, x; ¢, rNL)+y,@Il— Po. LL ELA. Bea a; 
~ | Xp (a), if xa. 


The system (x. 2 eer b P,, ie) may readily be seen to satisfy all 
the conditions 2.1.A-2,1.E and 2.1.F°, so that it defines a 
(non-cut-off) Markov process ea (When proving property 
2.1.F’ we have to use the relationship oS [2] = Nt. 3) 


We define the mapping a:2,;>E°™ as follows: if ?() ¢€[0, 
4)) is a function of 2, we put 


es o(t) for t€[0, A), 
eO=| 5 for t€[\.+ 00). 


We bring in the following notation: 


P= 2(Lf); h == inf (t,: x, =a}; 


, af 42> and x,€8, 
Q; tis pees ae Xnr - = if ¢ 
i a aca ta in remaining cases; 


J (Egy Xys eee Ene Xp» )=as bys. Spy agen Coo Ryton) 
+ 29; A RS SRE AO RESS 


The function g is clearly measurable with respect to x, ... 
Xn ++» Let ee aoe bis ove eS; + <>). If 9 (ty, Xy, soon bas Xn woe) 
—( then ath ty Xp -eealgs Myr. )=O0 and %=2 for all t,>2. 
In view of 6,1.B, theria is a function » of .”, defined in the 
interval [0, 7) and such that e¢j)=x, for all 4<), The 
function > =ao€ vevidently satisfies 9 (ts) = %: for all i, 
Condition 2,13,A is thus fulfilled for the process Xx and the 
space SF, Moreover, for all wEQ, 


(t,, Xt. cee Lia Xo,” ee a 
®; (4, Xt, eee ts Sn ...)=0. 
Therefore 
Gf: Fin Sew pe Mia ee VG OF ees ona bs Mieias) 


and by 6.1,C, condition 2.13.B is fulfilled, By theorem 2,8, 
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a process xX’ equivalent to X exists such that all its trajec- 
tories are contained in &Y,. The set —& is obviously inacces-~ 
sible for this process, In view of the normality of process 
X , the set — satisfies condition 3.8.B with respect to the 
process xX’ , By theorem 3.4, we can form a part X” of pro- 
cess X on the set FE, It can readily be seen that xX” has the 
same transition function as the initial process X, and all 
the trajectories of Xx” belong to #, The theorem is proved, 


Remark, Let all the trajectories of process X belong to 
the set % GQ, and let the function g(A; 4, 4, ..., ty, Xp +--+) 
satisfy conditions 6,1,A, 6,1.C and the following condition: 


6.1.B°. If o(é(¢E[0, 4))is an element of space o and 
Q (As ty DE). «+ Sar bEn) ---) =O, 


there exists a function 9€.9N_Y, defined in the interval 
[0,) such that ¢(¢j)= (4, for all ¢;€[0,i). 


Then there exists a Markov process equivalent to X such 
that all its trajectories belong to 9N.o, 


Proof, We take the process X, the mapping 2:Q,-Fl°™) and 
the function 7 constructed in the proof of theorem 6,1, Con- 
ditions 2,13. A’ and 2,13.B are fulfilled for the process Dan 
the function g and the spaces P=aF), Lo=2(LY). Hence, 
in accordance with the remark on theorem 2,8, a Markov pro- 
cess xX’ equivalent to xX exists, all the trajectories of 
which belong to Nn Fo + ~2he part X” of this process on the 
(inadmissible) set E is equivalent to X, and all the trajec- 
tories of X” belong to YNY. 


2, Conditions for Boundedness 


6.2. Let ¥ be an arbitrary system of subsets of the set 
E. We shall describe the function ¢(f)(#€[0, \)) with values 
from E as ¥-bounded if, for any T<i, there exists TCH 
such that 9(¢€I for all #E[0, T], The Markov process X= (x,, ¢, 
off, P, ,) im the measurable space (F, #) will be described as 
ys -bounded if all its trajectories are e¥ -bounded, 


We shall assume that the system ¥ is subject to the follow- 
ing condition: 


6.2.4. A sequence l,EI.¢... S',&..- exists, consisting 
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of elements of the s-algebra # and equivalent to F (see 
sec, 3,11)*). 


It may be remarked that a Markov process X is oF -bounded 
when and only when t=¢(#) (the function.{(¥) is defined by 
formula (3.58)). If the system ¥ satisfies condition 6,2.A, 
this equation can be written as follows: 

f= lim E.,) (6,1) 


hrm 


(the function (1) is given by (3.49)). 


Let '€ g and let A be a subset of the interval [0, 00). 
We put 


BO=is€r MM =Y lH ENF}. (6.2) 


tea 


The sets ¥,(T) and @,(1) are obviously disjoint. For a non- 
cut-off process their sum forms the space 2, This is not 
generally true for a cut-off process, 


Lemma 6,1. If X is a Markov process in the space (E, #), 
reg, and.A is a finite or denumerable subset of the inter- 
val |s, 4], containing the right-hand end 4, we have for any 
GE F: 

Polka DSPs. x; b,G)— sup Pu, y; 6, G). 
YET, wd (6.3) 


Proof, We suppose first that the set A is finite, and 
put 


ann x,éT}), af wE¥AM, 
—_ , it oC, (0). 


Obviously, « is a random variable independent of the future 
and s-past (2.=O,M)u",()). Lemma 5.4 justifies us in 
applying expression (5.8), so that 


*\This condition is satisfied by all normal systems and in 


particular, by systems 3,12,1-3.12.4 (see sec. 3,12), 
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P(s, x; b, G)=P, 2 {Q., xy € G} = [PG *.; 6, G)P,. (dw) = 
s (6.4) 


= f PG x5 OP. o(duyt f Pls, x5 b, O)P,, (du). 


¥a Q) ©, (I) 


Expression (6.3) follows from (6.4), 


If A is denumerable, we consider the sequence of finite 
sets A, tA(we choose the A, here so that they contain 4), We 
can apply (6.3) to each set A, , and on passing to the limit 
we find that (6.3) is applicable to the set A‘ 


6.3. Theorem 6,2. let the system ¥ © in the space (£, g) 
satisfy condition 6,2,4 and let the normal Markov process 
X = (x, Soff, P, Satisfy the following conditions 


6.3.4. There exists 6>0 and a sequence of sets E,€ # 
such that £,t& and for any n: 


inf su P(u, y; v, E,)=0. 
hee yee (4, y n) (6.5) 
0gv—u<d 


Then an %-bounded Markov process exists, equivalent to X, 


“Proof. We write Y for the set of all ¥-bounded functions. 
By theorem 6.1, the problem amounts to constructing a func- 
tion g(a; ty, 1, ..., ty, X, ...) Satisfying conditions 6,1,A-6.1.C. 


We take the sequence of [, which is equivalent to ¥ and 
for which condition 6.2.4 is satisfied, and introduce the 
following notations 


h(x) = sup (a: ET,) *), 
ROG Cp Bey -eake bye Bie Sa 
sup {k(x,):t,<A—e} for A< a, 
| sup{h(m)ite <=} 9 for k= oo. 


We put qQ ty Xp snes bye Kg «DRO AL AOArty Xp .... ty 
Xn ---)<(co for all e, If the opposite is the case we put 
QQ); ty Xp... ¢,, X,)=21. This function is clearly measurable 
with respect to i, x,...,%,... and satisfies condition 6.1.4. 


*\We put A(x)=0 if x€T, for all &, 
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Let A=‘t;,....4, ...} be amy sequence of numbers and ¥4%,..., 
Xn... any sequence of points of —, We consider the function 
g(t) (fE[0, 4)) given by 


xp if t=t,t<h 


Yo if te Ay ba ks (6.6) 


20= | 


where y, is an arbitrary fixed point of T,. It may easily 
be seen that 9€.% if 90; 4, 4, ....4,, X_, ---)==0. Condition 
6.1.B is thus fulfilled, 


We now show that condition 6.1.C is satisfied, Let A= {t,, 
.. fy, .-.}be an everywhere dense subset of the interval 
[s, co). We put for brevity g’=q(5; ty. Xt, ..-+fy, Xt -..), and 
define #° similarly, let e be any positive number. We 
take a sequence tending to +t 4<@<...<cuy<... EA, 


such that each of the numbers y4,—s, u,—4u,, ..., Us — Uy, oes 
is less than «/2, We put 
A,=[S, my], Se [ay, wy) ..., Ay [tyre oy) oe. 
A; = AA,, 
y= Loe (E) ——| is > u,}, ym) = Fa, (Pn): 
It will be observed that 
(2= 00} = Uf) Ye 4"). (6.7) 
k=O m=1 
In view of lemma 6,1, we have for each E, : 
P,c(¥e )>P(s, x; uy E,J— sup Pa, Y; Uy, Eq). 


yer» uEA, 


If <8 , it follows from (6.5) that 


to] 


lim sup P(u, V3 uy, E.)=0. 


Hence for any n, 


lim P, 2 {Ye} >P(s, x; ap, En) 


Mm PCO 


and therefore 
lim Ps, 2 (Vx\ ¥E] = Pa, 2 {Ye} — lim Py, (U} < 
™m foe) ™m > CO 


(6.8) 
<P (Ss, x; ty E)—PG, x; u,,E)=0. 
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It follows from (6.7) and (6.8) that 
Ps, 2 {2, = 00] =0, 


€ 


provided that 5=<3. It remains to notice that 


so that 


3, Conditions for Continuity from the Right and 


Absence of Discontinuities of the 
Second Kind 


6.4. Let X=(x, 6, ot, Ps,2) be a Markov process in the topo- 
logical measurable space (EF, @, #) and let 7(w) be any function 
subject to the inequality y<€. We shall say that the pro- 
cess X has no discontinuities of the second kind up to the 


instant 7 if the limits x;,,)(0) =lim Xu) and x;_»(w) = lim x, (w). 
wat 


exist for any wE€®2, t€[0, 7(w)). To. say that "the process xX 
has no discontinuities of the second kind" is the same as 
saying that "X has no discontinuities of the second kind up 
to the instant ¢", Let fT be any subset of E and ¥ any 
system of subsets of EF. We define —(T) by expression (3,49) 
and E(%) by (3.58). If the process X has no discontinuities 
of the second kind up to the instant §ET)E(%)), we say that 
it has no discontinuities of the second kind up to de e 
from [T (from KF ), Analogous meanings attach to the state- 
ments: "process X is continuous from the right up to the 
instant 7", "process X is continuous up to the instant 7", 
"process X is continuous from the right up to departure from 
rT", and so forth, 


6.5. Lemma 6,2. If the Markov process X is continuous 


from the right in the topological measurable space (EF, @, #); 
we have for any UC @@ and any xECU, fo0: 


° lim P(t, x; t+, U)=0. (6.9) 
hy 0 
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Proof, If a”, !0, we have 


[es] 
Ut {x= x, tran, € U} = 9. 


iD 


Consequently, aS m—-©o , 


(oe) 
Ps, «{U [Xt2ny € Ti} +0. 
But 


Py 2{t+nm CU} < Py | Ux nO}. 


a=m 


Therefore Pi, = {%tin, €U} =P(t, x3 t+», U) also tends to zero, 
which proves the lemma, 


6.6. We shall now suppose that the process is given in the 
metric measurable space (£, 5,9), the distance 0 (x, y) being 
@X P-measurable*), It follows from this condition (see 
lemma 1,4) that p(x, y)is a #-measurable function of y for 
any fixed x, The c-algebra # therefore contains the ¢- 
neighbourhood U,(x) of any point «, 

We put 


TO vcegtenn © x; t, U, (x)) (6.10) 
zx 


(here [ToF,2i>0). 
We bring in the following condition, 
The M(t condition, For any «>0, 


« jij en = 0. 
oo (¢) (6,11) 


Our main purpose is to prove the following theorem, 


Theorem 6,3. Let (E.p, #) be a complete metric measurable 


*)In accordance with sec, 18, p(x, y) is always a continuous 
function of the pair (x,y). Hence p(x,y) is @xg#- 
measurable if @ contains all open sets of the space (E, p) 
(see sec, 1.9). 


CONDITIONS FOR BOUNDEDNESS AND CONTINUITY 151 


space, the function.p(x, y) being # xX #-measurable, Let ¥ 
be a system of subsets of E satisfying condition 6.2.4, If 
a normal Markov process in the space (E, 5, #) is subject to 

the M(t) condition for every set TE ¥, an equivalent of this 
process exists which is continuous from the right and has no 
discontinuities of the second kind up to departure from F, 


6.7. We shall first of all prove a few lemmas, 


Lemma 6,3. Let X=(x, 6, oft, Ps.) be a Markov process in 
the metric measurable space (E,p, @). Letp(x, y)bea FX F 
-measurable function, Let Té gm. Then for any «>0,0<s 


gtgu : 
P, o {ET p (Xz, Xy) >t} <a (u—?). (6,12) 


Furthermore, if A is any given finite or denumerable subset 
of the interval [a, b] (s<a<b) and 


D=DiA= YU (o€T. HET. x.El. p(x» x) >4e}> (6,13) 


t,u€a 


we have 


Ps, 2(D) < 2a8 (6 —a). (6.14) 
Proof, By (2.9) 9 


P. > {*2€ Yr, p (Xp X,)>e} = f Py Ls {p (Xz, Xy,)>e} Py x (dw) — 
2,€P 


= | Pt x5 a T.CDIP,, »(do)<at(u—D, 
2,€T 


and expression (6,12) is proved, 


We turn to the proof of (6.14), First, let the set A be 
finite and consist of the points t,<4,<...<t, . We write 
t for the least among the numbers /f€A, satisfying the con- 
ditions x,€T, p(x, x, >2e (if there are no such numbers, we 
put t=max(s,C)). We put 


B={xg€T, p(xq xy) >t}, C= [p(xe Xp) De}. 
Clearly, DeByc, so that 
Ps, a(DI< P, 2(B)-+ Py, 2 (C). (6,15) 
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The random variable « is independent of the future and s- 
past, 


We apply the corollary of theorem 5.1 to the quantities 
=(0), (0) =t(0) 7% (©) = 9 (ES) and to the function 


1, if pO yo) Ss 
Ov w=| 0, if pOD yo) <e 


(in accordance with the remark at the end of sec, 5.9, this 
is justifiable because + 7, and %, take only a finite number 
of values in @.), We have 


P, ,(CloM)= Ms, o (fear x) lo} =FO x5 me = 
=F (zt, x23 % 5) (a.¢ 2, Ps =), 


where 
F (4, Bz) Uy, Ue) = M,,, yf (Xo, Xp,) a Pa 7] {p (Xo, Xy,) > e}, 
so that 


F(a, y; 2, b)=P(u, y; 6, U, (y)). 
Obviously, F(z. y; uw, b)<as(b--a) for any u€[a, db], yer. 


Consequently 


Ps, 2 (CoM?) < ab (6 —a) (a.¢.2., P 


8, x) 


and 
Py, (C) < at (6 —a). 


Moreover, in view of (6,12), Pe,2(B)<a.(6—a)., On substi- 
tuting these inequalities in (6.15), we get (6.14). 


If A is demumerable, we can choose a sequence of finite 
sets A,tA. Then Dr(A,)+Df(A), and we get inequality (6.14) 
for the set A by a passage to the limit, 


Lemma 6,4. Let X be a Markov process in the metric 
measurable space (FE. p, #) witha #X @-measurable function 
p(x, y) . Let Tré€g@, and let A be a finite or denumerable 
subset of the segment [a, 6] S[s, co). We put 
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A, = A, (A, T) = {there exist s,< 8:< -.. << SH < 
<% of A such that x, ET. x, ED. p(s, . xs,,) > (6.16) 
> 4 fori=1, 2,..., R}. 


Then 
Ps, 2 (An) < [2ab (6 —a)}". (6.17) 


Proof. We suppose first that the set A is finite and 
consists of the points 4:<4<...<4, . We take the random 
variable % , equal to max (s, 6) outside A, and given in A, 
by the equation 


ty (w) == inf (4,,: OC Axle, .. 0, tn.) }- 
We put 
"y= Max (f;, Tr) (i= 0, l, oo 04 n). 
Obviously, 
n 
Ary = JU, (a, € Yr, Xn, € r, 4, € I, p (%4,» X1,) > 4e] ‘ 


The characteristic function of A,,, is therefore equal to 
f (Saye coos Xx,,) » where 
(1. if y€T and there exist vw EF. 


FOr +++) In) = for which p(y, vs) > 4e, 
0 otherwise, 


The quantities «t, are independent of the future and s- 
past, whilst the ,...,% are ofM,-measurable, We have by 
the corollary of theorem 5.1.3 


Ps, 2 (AxsifoM,) =F (% Xep3 Mr + +21 i) (BeCo 2, .P ‘ 5)s (6.18) 


where 
F(a, Yi Uy, -.-1 Un) = My, yf (Xo ++: xy = 
= Py, [Dp «- ++ UJI: 
We conclude from lemma 6,3 that 


F(a, y; Uy, ..., 03) < 2ap(v,,— 0j) < Qap (b —a). (6.19) 
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From (6,18) and (6,19): 
Ps, 2 (Apes) = P,, a (AyAy 1) = 


(6.20 
= ‘ Ps, 2 (Azsifo42,) P,, (dw) < 2af (6 —a)Ps (Ay). : 
& 


Expression (6,17) follows readily from (6.20). 


If A is a denumerable set, we choose a sequence of finite 


sets A, tA and observe that At (An, rytA.(A, T). A passage 
to the limit shows that (6.17) likewise holds in this case, 


Lemma 6,5, Let the Markov process X=(x, [,o4#, P, ..) in the 
space (EF, @, #) satisfy the.M(I) condition (Ef). If +*,¢T 
for all 1€ [0. 7") and process X has no discontinuities of the 
second kind up to the instant 7, a Markov process X equiva- 
lent to X exists which has no discontinuities of the second 
kind and which is continuous from the right up to the instant 


n. 
Proof, We put 
= ea when <7, 
x, when y<ct<", 
We have for any s<#: 
eA MH) (Q >t xeH% Kyo} S (HET, x, # ro} (6.21) 


We notice that, if «>0,s<¢it<cu, then by theorem 2,1 and 
244,05 


Peel ED, p(t; x)>e)—= J Pye, (p(y x,)>8) Py, ,(dw)— 


az,€r 
= { Pd, x; u, U, (x,)) P,, 2 (dw), 
z,€0 


and in view of the M(I) conditions 
bey Ps 2 {XE p(x, Xy) >e} =0. 
But p(x, x,)—> p(x, X49) 88 ult , so that 


? 
{p (Xp Xz40) > &} S U ake P(e ee: 
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Hence 
Ps, » {xz€ T, p(%p X40) > eh< 
i iW pe >e 
< lim P,, c (* c a /P (* +2) aa } 


k=n 


< lim Ps, 2 {*€T, p (*e %,,1) a as 0. 


UN 


Since this is true for any «>0, we have 
Pen {xz,EL, x, xX449} = 0 


and in view of (6.21), Ps,2{y;#x,}=0 . An obvious conse- 
quence of this relationship is that X=(y, 6. At, P, ,) (the 

s-algebras Wt are defined in sec, 2,2) is a Markov process 
equivalent to process X, Evidently, X has no discontinui- 
ties of the second kind and is continuous from the right up 


to the instant %, 


6.8. Proof of theorem 6.3. In view of lemma 6,5, we sim- 
ply need to construct a process X equivalent to X and 
having no discontinuities of the second kind up to departure 
from %, We make use of theorem 6,1 for this construction, 


Let (tf) be a function in the interval [0, 4) with values 
from £., We put 


EI. 9) = inf {t: p@ ET} 
5(F.9)= supt (Tl, 9) j (6.22) 
rex 


(if p®ET for all t€[0, 4), we put ET. p)=)). We shall con- 
sider » belonging to the system . if the limits ¢(f-+ 0) 
and »(t—0) exist for each f€[0. (oF. 9)). 


Let e¢>0, A=({t, .... ty .--} G0, co), x... Xa... Ek , 
and let A be some interval, We put 


inf{t,:f,<d x,Er}, if ta<d x, E79 
bh (D) -| Garis Gr) for some a, (6,23) 
: in the remaining cases, 


a 2a d 
=) ap Ea (1), (6,24) 


156 THEORY OF MARKOV PROCESSES 


1, aif x«€Tandp(x, y)>4e, 
fate = 0 otherwise , 
and we define &,aQ;s ty %1 ...> tq, Xp, -..) aS the upper bound of 
the values of the sum 


h, (i) Xin) +h (Xig: Xi,) ++. rh, (Fen Xian)s 


when 2 runs through all natural numbers, the indicesi,,..., i,, 
being only subject to the condition that 4,<typ<...<t,, 
CANO, §) *), We put gQst. x, fa ty...) =0 If AAA, 
Xi. essa bas Xp -..)<C00 for any e>oQ and any finite interval A, 
Otherwise, we put gQ; ht, x, ..., ty Xp .--)=1. We note that 
the function g is measurable with respect to 4, %1,..., 4%, .. 
This follows from the fact that &,,, depends monotonically 
on e and A, and we can therefore consider only « and A run- 
ning through certain denumerable sequences when defining g, 


The function ¢ clearly satisfies condition 6.1.4. We shall 
verify that it also satisfies conditions 6,1,B-6.1.C, Let 
A= {t,,..., 4, ...} be an everywhere dense subset of the inter- 
val [s, co), If qi; 4, x, ..., ty, Xp...) = 0, Whatever the 
monotonic bounded subsequence 4,, 4, ...,t,,...<§ of the 
sequence 7,,..., t, .... the corresponding sequence of points 
Xi,» ---, %j,,-..Satisfies the condition: for any e>09Q there 
exists n(e) such that P(*iy %i,,)<e for m,n>n(e) . In view 
of the completeness of space E, the sequence Kgyeces Kags oes 
is convergent. 


We define the function ¢(¢) (¢€[0, 4)) by the formlae 


Xj; if t=4,€AN[0, 2), 
e(j)={ lim ot). aif t€[s, )nA, 


Uyt,weA 


(6.25) 
e(s) if +t€[0.s) or if FE [E, NN, 
It is easily seen that §(F, p)<t, and if ¢,....%,...is any 


monotonic bounded sequence of points of [0, §) » the limit 
line (Z,) exists, The function » therefore belongs to the 


class #, Condition 6,1.B is satisfied, 


*)If such indices can not be found, we put A, = 0 
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We now show that g satisfies condition 6.1.0, Let ayo, 


A={[h, ..+) fy... }E[s, co). We put 
*_ , 
g =9(5; th Xtiv eee Ens Xt r+), 
AL =A S tis Xt, ; eee, Eas ma ’ -++), 


Am =[s—-+- ms, s+-(m--1) 8] (m=0, 1, 2,...). 
Clearly, 


{g*>O0seS h* == 00 
UU ~, An : (6,26) 
Let T,o...G¢7,4... be a sequence of sets as described in 
condition 6, 2 A, The en EA (Tr) tt and therefore, for any e>0 
and any k, 


re 


Ay (ANA, Ty). 


In 
re 


where A; is defined by expression (6.16), It follows from 
this that 


Pe, a { % hg =ool< < lim Ps, @ {ARAN Any: T)}. 
By lemma 6.4, 


Pez {Ae(AN dm, Ty)} < [2a @)}*. 
Consequently, 


P,2{%. =o} < [20h @)*. 


m 


By the hypothesis of the weporems for any e>0 there 
exists a 6>0 such that gf, O<z . We have with this choice 
of 8, 


Pol M =] <(3)", 


and, since this is true for any &, 


Pe, x { Ay , 4, == OO} = 0. 


On combining this equation with inclusion (6.26), it may 
be seen that P, .(¢*>0)—=0 , and the theorem is proved, 


6.9. Remark 1, Let (E. p, #) be a metric measurable space 
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satisfying the conditions of theorem 6,3, and let ¥ bea 
system of subsets of & equivalent to a sequence of closed 
sets T.ar,c¢...¢7°r,c.... If the normal Markov process x 
in the space (E, p, #) is #-bounded and satisfies the M(T) 
condition for all Té¥, a Markov process equivalent to X 
exists which is ¥ -bounded, continuous from the right and 
devoid of discontinuities of the second kind, 


Proof, We define Y as in the proof of theorem 6,3, and 
write Yo for the class of all ¥ -bounded functions of One 
We notice that, if $€.% and x;=—(¢,), the function o(¢t) given 
by (6.25) belongs to N.Y. It may be concluded from this, 
in view of the remark on theorem 6,1, that a process X 
equivalent to X exists, all the trajectories of which belong 
to YNHo. We have t(#)—% for this process, which proves 
our statement. 


Remark 2, Let (F, p) be a c-compact complete metric space, 
# the s-algebra generated by all open sets of (E, p) and P(s, 
x;t,T)the normal transition function in the space (E,p, #). 
We assume the existence of a sequence of closed sets Tl, +E 
such that: 


a) the M (~,) condition is fulfilled for each PY, 3 
b) there exists 0<@ such that, for all m, 


lim su : ie 
ne, SUP P(t Ys UV, Tm) =0. 
ao yer, 
Jv-u} <s 


Then there is a Markov process with the transition function 
P(s, x, 4, T) which is (T,,! -bounded, continuous from the right, 
and devoid of discontinuities of the second kind*), 


If condition 5,18.A is satisfied, or if P(s, x; t, T) =P(t—s, 
x, T), where P(t, x, T) satisfies requirement 5.19.4, this pro- 
cess is strictly Markov, 


Proof, By theorem 4.2, a Markov process X exists with the 
transition function P(s, x;t,T). We put & =({T,}. By theo- 
rem 6,2, we can construct an  -bounded Markov process 
X” equivalent to X. This process satisfies the M (I) condi- 
tion for every n, In accordance with remark 1, a process X 


*)Needless to say, 1t is sufficient to require that the M(4 


condition be fulfilled instead of condition a)-b), 
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equivalent to xX’ exists which is F -bounded and continuous 
from the right and which has no discontinuities of the second 
kind, When condition 5,18,A or 5,19,A holds, this process is 
strictly Markov by theorems 5,9 and 5,10, 


4, Jump-Type and Step Processes 


6.10. The Markov process X= (x, C. At, P,. x) » in the measura- 
ble space (£, #)is said to be a jump-type process if, for 
each w€@, 1€{0, C(w)) there exists ¢s0 such that ~;,,(w) = x; (w) 
for all 0<’%<8, The number ¢ is called the instant of 
jump of the trajectory +,() if the sequence ¢,+¢ exists such 
that x1, (©) #x,(o) (29=1, 2,...), The instants of jump may 
easily be seen to form a denumerable set, If, for any o, 
this set has no limit points inside the interval |0, C(w)) , 
we shall describe X as a step process, 


We introduce a discrete topology in the space (EF, #), tak- 
ing for @ the class of all subsets of £, The class of jump- 
type processes in (£. @) is easily seen to coincide with the 
class of processes continuous from the right in (£. @, #)3 
the class of step processes coincides with the class of pro- 
cesses continuous from the right and devoid of discontinuit- 
ies of the second kind, It follows from theorem 5,10 that 


every stationary jump-type Markov process is strictly Markov. 


The topological space (E, @) is metrizable, The correspond- 
ing metric is given by 
0, if x«=y, 


p(x, y=| i. Be" ces 


The following is a sufficient condition for the function 
p(x, y) to be measurable with respect to @X #: 


6.10,A, The subset of space EXE consisting of points of 
the type (x. x) (x€E) belongs to the c-algebra @X#. 


The following theorem is obtained by applying the results 
of article 3, 


Theorem 6,4, Let (E, #) be a measurable space satisfying 
condition 6,10.A, and let xX be a Markov process in (£, #) 
subject to the condition: 


6.10.B, The convergence is uniform over t>0, «€E: 
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lim P(t, x; t--k, E\x)=0. 
hyo 


Then a step process exists which is the stochastic equivalent 
of process X, 


We suggest that the reader define a process which is jump- 
type (or is a step process) until departure from the system 
F, and state the analogue of theorem 6.4 for such processes, 


6.11. Examples, 6.11.1, Let F=({1,2,..., ”,...} and let # 
be the system of all subsets of &. Im accordance with 
4,2.2, every transition function in the space (E, #) can be 
specified with the aid of a system of functions ij(5. 1) O<¢ 
s<t). Condition 6,10.B is expressed in terms of the p,;(s, 2) 
as 


lim sup 2 Pig (ts t+-h)=0. (6.26 *) 


hyo ti 


In the stationary case, this amounts to requiring the uniform 
convergence (in regard to i)of: 


lim .-(h) — 
lim 2 Pig (2) =0. 

Furthermore, if the set F is finite, the uniformity require- 
ment is satisfied automatically, and condition 6,.10.B is 
equivalent to the requirement that p;(h)>0 as h}0 for any 
ixéj . 


We can conclude, on taking into account the fact that the 
conditions of theorem 4,2 and condition 6,10.A are fulfilled 


in the present case, that there is a corresponding step pro- 
cess for every transition function subject to condition 
(6.26°) in the finite or denumerable set. 

6.11.2, The Poisson transition function (example 4.2.6) is 
readily seen to be complete and to satisfy condition 6,10.B, 
There will therefore be a non-cut-off step process correspond- 
ing to the Poisson transition function, if the phase space is 
subject to condition 6,10.A and a topology satisfying the 


requirements of theorem 4.2 can be introduced into it. We 
shall term this a Poisson process, 
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5. Continuity Conditions 


6.12. The Nf) condition formulated below will be shown to 
play the same role in investigating the continuity of a 
Markov process as is played by the M(Y) condition in criteria 
for continuity from the right and absence of discontinuities 
of the second kind, 


The N(Y) condition, For any e>0 , 


af (3 
peed a 
sy0 


= 0, 


We shall make use of the following lemma when deducing 
criteria for continuity, 


Lemma 6,6, Let X:=(x, €, off, Py, ~) be a Markov process in 
the metric measurable space (E. p, #) (witha #X #-measura- 
ble function p), let the set TC @, and let the subset A of 
the segment [s. 7] be finite or denumerable. We write §={ 
for the strict lower bound of elements f of set A for which 
x,EéT, and put 


Q= QE" (A) = JU, {E<E 2<E p(xp £)>4e}. (6.27) 
(tures 


an =e 


Ps, 2 (Q)<21T —s] (6,28) 


Proof, We choose the points 4<&%<...<4,€[s, T] such 
that 


u,— $= Wh — a... = Uy, —~ Uy _, = 4, 0< T—u, <4, (6.29) 


and put 


Ay =[Ss, uy], Ay = [ay, ug], Ap = [at, al, oso, Ay y= [Uy_y T), 
D;= Dp (ANA; @g=1,..., k—1), 


where the event Dj is given by expression (6.13). Clearly, 
Qc U D; ° Therefore 


tml k-1 


P, 2 Q< = P,, » (Di). (6,30) 
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Since each of the sets ANA; is situated on a segment of 
length 2¢, we have by lemma 6,3: 


Ps, 2 (Di) < 2a (28), 


and in view of (6,30): 
Ps 2(Q) < 2 (& — 1) ap (28). (6.31) 
T—Ss 


But, by (6.29), R<T—s, Therefore p—1 <k< ——* =, 
and (6. 28) follows from (6,31). 


6.13. Theorem 6,5, Let (£. p, ) be a complete metric measur- 
able space with # X #-measurable metric p, Let [ bea 
measurable set in the space (E, p, #@)and X a normal Markov 
process subject to the N(I) condition, Then a Markov process 
equivalent to X can be constructed which is continuous up to 
departure from [ and is such that, for every wC@, either 
é(r, w)=2(v) or lim x,(w) exists and belongs to the boundary 
r of the set r’*er) 


Proof, Let ¢(t) be a function in the interval [0,4) with 
values from £—, We define the quantity &(9)=&(I. %) by expres- 
sion (6.22) and we shall say that the function 9 belongs to 
the system 2 if: 


a, ) o(t) is continuous in the interval 0, &(¢)) ; 
a5) either §()=2, or A g(t) exists and belongs to I’, 
Let T>0,8>0, A= {t, ..-5 Ens») &[0, co), XH... One EL 


We. define the function §=%(1) by. expression (6. 23) and put 
&7 — min (é, T)3 


h; AS ? > eees > — ° e 
TAS by, X ta» Xp» ---) tot eer (%;, ©;) *); 
| t-8y[ <2 
he=limh, ,, h=limhy, 
ayo UT Paco . 
x,y), Jj 
£0 nt ym{ 9 HE PCD ER IEF 
-+-co in the remaining cases; 


*)If the set for which the upper bound is taken is empty, 


we put 
pu hy A; th Xyy e205 Ens Xn «os =O . 
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8: Ae hiv Ky ans E eee 
inf Ms 43.3, %3), Af F<}, 
_| mee its *s) =? 
| 0, sf = 
BAS by Has ee er Lar Xa oe DS lim 8 OE Lys Ray, ed dat eae). 
The function g=—A-+g satisfies condition 6,1,A and is 
measurable with respect to i, x, .... %_--- We prove that it 


also satisfies conditions 6.1,B-6,1.C, 


Let A=({t,, .-., ty -.-} be an everywhere dense subset of the 
interval [s, 00), 4€(s, 00) and let g(; ty. Xp» --- tx, e..) =U 
We suppose first that —§<} 3; we choose 7€(é,4). It follows. 
from the equation Ag(h; ty, x1...) tas Xn ---) == 0 that, for any 
e>0, there is a 6>0 such that p(x. x))<e for t,t,<tTRE 
and |t;—t;|<6,, This means that the function «© given by 
o(t;)=x;in the set Af[s.&) is uniformly continuous in this 
set and may therefore be extended (uniquely) to the contin- 
uous function ¢(f) in [s,&), which has a limit as tt&. We 
complete the definition of this function by putting 


fo(s) for O<t<s, 
o(t)—=) xj, if t=4,€[5, 
| x, if t€[& ANA. 


The function o/(t) satisfies the condition ¢(¢;))—+*; for all 
t,<h , and condition a,). Moreover, &(v)<£ and condition 


a) is obviously satisfied if §(¢)<&. On the other hand, 


if &(9)=& , it follows from the relationship g(\; ty, x.---. ty 
Xnr -.-)=0 ee et ee) T)=0 for any §>0. We put 


om ee) . Evidently, p(z,T)== ep. M=0. Consequently 
z€I”, This proves, in fact, that the function » satisfies 
condition a5) and therefore belongs to .f, 


We now suppose that §—}. Since 
h(a; ti Xys wees Las Xno os .)=0 


for all 7, the function 9, defined in the set A by the 
expression o(f;)=.x; is uniformly continuous in the intersec- 
tion of this set with any segment [s, T/c[s,4). The function 
may be extended uniquely to a continuous function in [s,)). 
On extending it to the interval [0,s) in accordance with the 
expression ¢(t)=¢(s)(¢€[0, s)), we evidently get a function of 


2. 
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Turning to the proof of condition 6,1,.C. we write for 
brevity b+ =i (1), ge = 25 tis Xt, ee Es od ) and similarly 
define ,”, Ar. h;, 7 and gq ‘ 


Let A=‘t,.... ty ---} be an everywhere dense subset of the 
interval [s,oo). It will be observed that, for any positive 
z; 6, T : 


{ar 4e} S [Ar > 4e} QF (ANI. TI}, 
where Qe is defined by expression (6.27 ) . Lt follows fron 
this, in accordance with lemma 6,6, that 
ola 


P, 2 {ar > 42} <2(T—s) 


If we let 3-.0, we have from the N . condition: P, (hp 
> 4e) = 0 e 


Since rppthas Ttoc , we have 
oc oo “ 1 
hy > ’ 
UU {0e> 2} 
whence it is clear that P, ,(h*>0)=0. 


We now represent the set A as the sum of finite sets 


b,) MENS... CAPS v5 


iy 
do) Mis — at <8, (R=) .2, er m, — 1) 9 where 8,40 as 
IL——> WW 6 


We agree to write for brevity x<® instead of Xin es 
We bring in the random variables: 
Pa==sup {ar:x°CT for j=1, 2, seer Rh, 
Ya = inf [ue: xR E Dy. 
We notice that, for any T>0, 


Py % Jee ee <T}S UY (xPer. olen. xp.) >el. (6,32) 
ur<T 
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But in view of (6,12) and b,), 
Pee (Er: p(x}, xP.1) 28 h< a (411-4) < 2, (8,)- (6.33) 


On the other hand, by b,) , the number of points cf sets A 
located in the segment [s. T] does not exceed 3(7 “o—) 3(7—s) 11, There- 
fore we have from (6.32) and (6,33): 


Ps, 2[P (ngs Hq) 26 Bn <T] SCL, (6.34) 
where C=3(T—s)+6, « 


We now write =<, for the greatest of the elements of set A” 
less than ®. It is evident that =<p,< #<v, and vy, |e 
as n—co, On the other hand, x, t& by virtue of b 5) There- 


fore %.—t™,|0, and all the more, p,—7,-0. We beeies that, 
for any T>0,e>0,8>0 and any sufficiently large .n: 


(g* > 5e} S [<p (xn, %,) > Se} S 
S(T <ocoO}U (tn a, > B!U 
U {& < T, Yn — Fn <3, p(xXz,: xy )>5e) & 
S (TE < oO} U {oe — Tn > 3U 
UQr'(A* Ns, T) ULE < T, p(Xa,+ %y,) > eh. 


This gives us, using lemma 6,6 and inequality (6,34): 
P., 2 {g" > aS Pee ST Soo Raa =e 2 Oy, 


$2(7—9 BO 4c Oe) 
If we first let « tend to infinity, then 6 to zero, " and 
agp T to infinity, we now get P, .{g*>5e}—0. Thus 
zig >0}=0, The relationships proved above: P, , {h*> 0} 
= 2,018 > O}==0 show us that P, .{g*>0}—=0. Condition, 
6.2,C is fulfilled and the theorem is proved, 


6.14. Theorem 6.6, Let X be a Markov process subject to 
the requirements of theorem 6,5 in the space (£.o, 9) . let 
FF be a system of subsets of space (E, p, g) such that the N (1) 
condition is satisfied for each TES. 


Then a Markov process X can be constructed which is equi-~ 
valent to X, is continuous up to departure from oF and has 
the following properties: for every of © there is either a 
PEF such that x-(o)ET for all #€[0,f()) , or else for every 
['€¥ there is at, tE(@¥%. w) such that Xt, @eér. 
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If the process X is ~-bounded, X can be chosen so that 
(oF )= 5 - 


Proof, Let f,, be a sequence of sets satisfying conditions 
3.12,4-3.12.C, We say that 962 if o is continuous until 
departure from # and either x,€T,; .for some mand all $e (0-4); 
or there exists the sequence /, t:(¥,¢) Such that o(t,)ET,. 


We construct for each set [,, a function Im 28 described in 
the proof of theorem 6.5, We shall write ¢) for the result 
of Substituting hand xz, for x, in 9m. We put a= Dam 9 * 


= 30, The function g is clearly measurable with ‘respect 


to Xp...) Xp --- and satisfies condition 6.1.4. When prov- 
ing theorem 6,5 we showed that Ps,2{7,, >} =0 . Hence P, .{q° 
>0}=0 and condition 6,.1.C is satisfied, It remains to 
prove condition 6,1.B. 


Let A==[t, ..., tg ..-'S[s, co), 4E(s, 00). We put 
En = th (Im): c= th (F)=limtn 


where (1) is given by (6.23). We assume that 9Q; 4, x, ..., 
tas Xn -+-)==0 80 that ImQs ty Xp ..-. ty Ly...) == Ofor any m, 

We consider a function ? defined in the set A by the equa- 
tion o(f=+*,. This function is uniformly continuous in 
Afi{[s, K] for every K€[s, §) and can therefore be uniquely exten- 
ded to a continuous function in the interval [s.&) . We put 
¢(f)=9(s)for ¢€[0,s) and define » arbitrarily in the interval 
Gh) ty 208 the proviso that o(f;))—-x«; for all i, Let 
En ==" (ne ©) E(¥.9). We show in the same way as when 
proving . ce 5 that, if <A’, then ne ?@) exists and 


belongs to Ta. It follows from condition 3 2.0 that Tm 
GE\I,. Therefore En <Euat ‘ Hence either §,—. for a cer- 
tain m, or tichic ... cin... <&. It may easily be seen 
that in both cases the function 9 that we have constructed 
belongs to Y,. This completes the proof of the first asser- 
tion of the theorem, 


It can further be seen that, if » is any ¥-bounded func- 
tion in 2g and if x,=%(4), the function constructed by us is 
also #-bounded, The second assertion of our theorem follows 
from this in accordance with the remark on theorem 6,1, 


6.15. Corollary. let (F,») be a s-compact complete metric 
space, # the s-algebra generated by all open sets, and 
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Ps, x; Ta transition function in the space (E,5, #), 


Let # be a normal system of subsets of space (E.». F) 
satisfying condition 6.3,A, and let the N(I) condition hold 
for every TE ¥. Then there exists a continuous ¥-bounded 
Markov process with the transition function P(s, x: 4,T). 


If condition 5,18.A holds, or if the transition function 
is stationary and satisfies requirement 5.19.A, the process 
is strictly Markov, 


The proof of this corollary follows exactly the same lines 
as that of remark 2 on theorem 6,3, 


Remark 1, The corollary obviously still holds in the case 
when the N(E) condition is satisfied, 


Remark 2, If the transition function P(s, x; 4,T) is complete, 
we can evidently choose a non-cut-off process as the process 
described in the corollary, Similarly, a stationary process 
can be chosen if the transition function is stationary, 


6, A Continuity Theorem for Strictly Markov Processes 


6.16. Theorem 6,7. Let ef *) be a system of open measurable 
sets G,CG.f... &G,¢... in the metric measurable space(F, p, #) 
with #X #-measurable function p(x, y), and let y—(x, £, 
ofM},P, .) be a strictly Markov process in the space which is 
& -bounded and continuous from the right**) and satisfies 
conditions M(G,)(n=1,2,...). Letu<n<c...<€%<...be an 
arbitrary sequence of random variables independent of the 
future and s=past. We put 


t(w) = J (w) (w € 2) 


o. = fect, (6.35) 


*)The theorem obvicusly still holds if #¥ is any set system 


equivalent to the sequence {G,}. 

**)Tf process X were not continuous from the right it could 
be replaced by an equivalent process that was continuous 
from the right, in accordance with remark 1 on theorem 6.3, 
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Then 
Xen (wu) Xv) (oo B,, Pa, 2)- (6.36) 


Proof, Let :(G,) be the instant of first departure of the 
trajectory from G, (see sec. 3.8). Since process X is ¥- 
bounded, we have {7 <§(G,)} t2., and consequently 


{ <5 (Gi): aa H x,} t {2., Xa, rad x,}. (6 -37) 
Furthermore , 
[F<2OD. x] =U (6.38) 


mm 


where 


=U Rico x)>o, +<t@)}. 


N=1n2>N 


We choose an arbitrary £>0 and put 
fé, ) aoe ero nye (++) ke, (Xt4u) Pp Xeru) 
(t€[s, co), o € Q,). 
Obviously, 


ce YJ 1) {rem wo) >=}. (6.39) 


M=1N= 


Since X is continuous from the right, whilst the sets G, are 
open, we have 


Fé, o)= sup Xq (4) Zg (e4u) Pp Keen)» 
UGA k k 
where A is the set of all rational numbers of the interval 
{0,”2). It is clear from this that the function /f(f,) is 


measurable with respect to the c-algebra 3°, and by theorem 
5.28 


Ps, 2{F (tw ©) > 5] = Ms, oF (ty Xn) (6.40) 
where 


F (wt, Y)=Py,y{ f(a, w) > 2h. 


By lemma 6,3, we have for any yCG;3 
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a 


F (us y) < 228 (h). (6.41) 


We obtain by combining (6.39), (6.40) and (6.41): 
1 
Ps, 2 (Cin) < 227 (A): 


1 
In view of the M(G,) conditions, «, > aS h-+0, so that 
Pas (Go) A . On taking into acedant (6.37) and (é. 38), we 
get (6.36). 


Remark, Theorem 6,5 holds for any strictly Markov process 
which is continuous from the right and satisfies the M(E) 
condition, For we can put in this case ¢G,—=...=G6,= 
=E, 


6.17. We shall in future describe the Markov process X= 
(x,.%, Mf, Ps,2)@S quasi-continuous from the left if, whatever 
the random variables 7, t:, .--, T, -.-independent of the future 
and s-past, and whatever the subset © of set Q, the rela- 
tionship 


t(w) ft(w) <f(w) — (w EB), (6.42) 


leads (for every x€£) to the relationship 


X_ —>xX, (a.¢.9, p 


2 Ss. x) 


We arrive at the following proposition by using theorems 
6.7 and 5.9 in conjunction with remark 2 on theorem 6.3 
(sec, 6.9): 


Theorem 6,8, Let (E, p) be a c-compact complete metric space 
and #@ the s-algebra of all Borel sets of the space (E, p). 
Let ¢¥ ={G,} be a sequence of open sets such that the closure 
of G, is contained in G,,, and G,tE. The transition func- 
tion P(s, x; t,T) in the space (E, p, #) will be assumed to 
satisfy conditions 5.18.4, 6.3.A and M(G,) (1n=1, 2,...). 


Then a process can be constructed with the transition func- 
tion P(s, x; t, T) such that it is strictly Markov, continuous 
from the right, quasi-continuous from the left, devoid of 
discontinuities of the second kind and #-bounded. 


Remark, If the transition function P(s, x; t, T)is stationary, 
conditions 5,18.A can be replaced by condition 5,.19.A, i.e. 
the requirement that the transition function be of Feller's 
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type. 
7. Examples 


6.18. The phase space in all the examples below is a s-con- 
pact complete metric space, whilst the o-algebra # is gene- 
rated by the system of all open sets of this space. We can 
therefore base our discussion on the corollary of theorem 


6.6, 


6.18.1. We have for the transition function P(s, x; t, N= 
Xp [x-+ u(t—s)](see sec. 4.2.1): 


af (6) = at.(8) = 0, if a<K. 


In accordance with the remarks at the end of sec. 6.15, there 
is a stationary non-cut-off continuous process on a straight 
line corresponding to this function. We shall describe this 
process as a determinate motion with velocity v. We can 
easily show that, for the process, 


Pi{x,=x+2t for all t>O0j=—1, 


6.18.2, We have for the Wiener transition function con- 
structed in sec, 4.2.3: 
Pemairaaen ” 2 
P(t, x; t-+-h, U, (x)) =[2eh] 2 f e 2% gz, 
lz|>e 
where 2? denotes the scalar square, and |z| the length of 
vector z, We get from this, after some simple calculation 


Pix; th U, C))= eaF (Fz): (6.43) 


where c, is a constant, and 


ipo} 


r? 
Fir)= f ez rv—1 dr, 


r 


It follows from (6,43) that 
ae @) = cnF (==). 
By L'Hospital's rule, Fir) pR-2p-5 as r—->cso, so that, as 
6103 
n—-2 2? 


at (6)~e,s"-28 2 9 2, 


It is clear from this expression that the N(E) condition is 
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fulfilled, In accordance with the remarks at the end of sec, 
6.15, we can conclude from this that there is a stationary 
non-cut-off continuous Markov process which corresponds to 
the Wiener transition function, We describe it as a Wiener 
process, It may easily be seen to be of Feller's type. It 
is therefore strictly Markov (see theorem 5,10), 


6.18.3. We shall now investigate the transition function 
described in sec, 4.2.4 in the interval [0, 0). We notice 
that, for any a>0,x>0, t>0, A>0: 

Pit.x; t--h, (a, co))= , 
1 > 1 2x 
= Vax vi é aS Te i (6,44) 
lyVn-al<ez 

It follows easily from (6.44) that the system ¥ of all 
sets [2, 00) (a>0) satisfies condition 6,3.A.([1,5. ) can be 
taken as £,). System ¥ is obviously normal,'”Simple calcu- 
lation similar to that of 6.18.2, shows that, for «<a, 

1 2 
We, 20) (8) ~ ce-332 9 25. 


TheN (T)condition thus holds for any T€%. and by the corol- 
lary of theorem 6.6 (see also remark 2) an #-bounded contin- 
uous stationary Markov process exists with transition proba- 
bilities 4.2.4. This process is clearly of Feller's type and 
is therefore strictly Markov. It can be shown that, for any 
s>0, x€E: 


lim x} (w) — 0 (a,c ° 25 P,. ey P 
tags (w) 


We may describe the process as a one-dimensional Wiener 
process with a break at the point 0*). 


6.18.4, For the transition function of sec. 4.2.5 we have 
at(d) wee -= , and some continuous Markov process there- 

fore corresponds to the function. In accordance with remark 
2 (sec. 6.15), we can choose a non-cut-off stationary process 


for the process, We shall refer to this as a one-dimensional 


Wiener process with reflection at the point 0. 


6.18.5. We shall now construct an example of a continuous 


*\What we have described is not in fact one process but a 


whole class of processes, For the sake of clarity we can 
choose from among these the canonical process which is subor- 
dinate to all the rest, 
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t:! 
f 


Markov process which is not strictly Markov, 

_ We texe a stationary transition function on the real line 
E=(—-., +0°)with the usual metric p(x. y)=|y—x], the func- 
tion being defined by 


a8 (x—y)? 
VoGlo fT dy, if x-=0, 
Pils, 222 T)= ¥ 25 ¢—S) I is ( Fa 


Lp (*); if «=0. 
this function is complete and satisfies the N(E) condition, 


herefore corresponds to some non-cut-off continuous 
icnary Markov process X=(x,, of}, P,, 2) 


vt 
os 
ct 
pa 


The instant <—£(G) of first departure of the trajectory 
from the set G=E\ {0}is a random variable independent of 
the future and 0-past, For, if A is a denumerable set 
everywhere dense in the segment [0, ¢] , we have 


(>H=U  {ixel> a} com 


We put w€A if there exists a ¢ such that x, (w)—0 for all 
umpt, It may be observed that, if «+0, Py) .(A)=0. This 
follows from the inclusion A cl ie ='p] and the equation 


Po, z iX, = 0} = P (0, x; on, x)=0. apis 
On the other hand, Pp, ,(A)==-1. For 


AZf){x,=0}, 


rEA 


where A is the set of all rational non-negative numbers, 
Since for every rs 


Poo {xp = 0} = PO, 0; 7, O*)=1, 


we have 


and all the mores 


Py, 9 (4) = is 
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It may easily be seen, moreover, that J,A—Afor any u, so 


that 6.A =A e 


If the process X were strictly Markov, we should be able 
to write, in accordance with (5.40), 


Po {6, A} = Mo, xPo, a, (A). (6.45) 


But x,—=0, so that the right-hand side of (6.45) is equal 
to unity for any x*), At the same time, the left-hand side 
is equal to zero for x+#0. 


*\Tt is easily proved that Po«(%.}=1-for all x, 


ADDENDUM 


A THEOREM REGARDING THE PROLONGATION OF 
CAPACITIES AND THE PROPERTIES OF 
MEASURABILITY OF THE INSTANTS 
OF FIRST DEPARTURE 


1, A Theorem Regarding the Extension of 
Capacities 


1. A compact Hausdorff space with a denumerable base*) is 
referred to as a compact. 


Locally compact Hausdorff spaces with denumerable bases 
will be described as semi-compacts. Some properties of such 
spaces will be listed, 


1.4. A semi-compact is a o-compact space, 


1.B, A topological space is a semi~compact when and only 
when it can be got by discarding a point from a compact, 


1.C, A compiete metric can be introduced into any semi- 
compact. 


1.D. Hach closed and each open set in a semi-compact can 
be written as the sum of a denumerable number of compact sub- 
sets, 


Proof of 1.A. Let @S C be a denumerable base of the semi- 
compact (E, @). We write @’, for the subsystem of system @ 
consisting of all the elements of @ with compact closures. 
Let x€E&., In view of the local compactness of (F, @), there 
exists a neighbourhood U of the point x with a compact 
closure. Since @ is a base, there exists G€ @ such that 


*)A subsystem @ of a system Cis said to be a base of the 


topological space (E£, @) if every set Te@ can be written as 
the sum of elements of @, 


L74 
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x€Geu. Clearly,G€@’. Thus every point x€£ belongs 
to some set GE @’, and @’ is a covering of space FE, The 
closures of the elements of @’ form a denumerable system of 
compacts adding up to the sum F, 


Proof of 1.8, Discarding a point from a compact evidently 
leads to a semi~compact, We show that a compact can be got 
from any semi-compact of (£, @)by adding a single point ca, 

We put F, Ev {a\and write @, for the system of all subsets 
appearing in C and of all sets of the form {a}U{E\T}, where 
Yr is any compact subset of space (EF, @), It may easily be 
seen that (£;, @:) is a compact, 


Proof of 1.0. In view of 1.B we can assume that F—EF,\ {a}, 
where (£;, @,) is a compact, into which some metric 6,(x, y) is 
introduced*), This metric may easily be seen to be complete. 
The expression 


p(x, =x, y+] pi(% @)'— p(y, a7") (%, YEE) 
is readily seen to define a complete metric in E, 


fev) 
Proof of 1.D, In accordance with 1.4, E=[J£,,, where 


mol 


the £, are compact sets, If [ is closed, we have 


r=UTlEn 


m=1 


and the TE, are clearly compact, Now let G be open, and 
let p(x, y) be any metric in (£, @) (existing by virtue of 1.C). 
We put 


Pa={ yp (x, E\G) >>}. 


Obviously, [,, is closed and 


The sets [,,F,, are compact, 


2. We shall write # (£, @)for the class of all compact 
subsets of the topological space (E, @), #.(E. @) for the class 


*) The proof of the metrizability of any compact can be found 


e.g, in Ref, 25, 
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of all subsets which are the sums of a denumerable number of 
compact subsets, and 2%.s3(E. C)for the class of all subsets 
expressible as the intersections of a denumerable number of 
sets of #.(F. C). 


We shall describe a subset I of the space (E, @) as analy- 
tic if, in some compact (E’, @’) a set BEc#3(E’, @’) can be 
constructed which permits a continuous mapping on to [. 


Lemma 1, Every Borel set in the semi-compact (F. @) is an 
analytic set, 


Proof, Let & denote the class of all analytic sets in 
the space (€, @). 


1°, We show that, if T,€C% (n=1, 2, ...) and 


=— U |e 


m= 1 


then TES. 


By hypothesis, there exists for any n a compact (E,, @,) and 
a set B,C #3 (E,, @,) admitting a continuous mapping f, in 


he 


Let - 
E= (JE, 


n=1 


and let @ be the.class of all subsets of the set E express- 
ible in the form (Un » where U,C @,. The topological space 


(E, @) is a semi-cémpact, so that (see 1.B) it can be got by 
discarding a point from some compact (E’, @’). We put 


B=U By 
n= 1 


and define a mapping f:B-+E by the expression f(x)=f,(x) 5 
if x€B,, It is easily verified that BE #(E’. @’) and f is 
a continuous mapping of B on to I’, 


2°, We now show that, if 
T,CH (a= 1, 2, ...) 


and oo 


r=f] ry 


Timi 
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then TE, 


Let (En, Cn) B, and f, have the same meanings as in sec, 1°, 
The space 
(E’, C’)= 
=(E, xX Fy... KEK oe C1 KX Co®% ee KX CaX-o) 


is a compact. The set 
B=B,XBX...xXB,X... 


is the intersection of sets EF, x... XE,_1XBaX Eni X.. CK 
(E’,@’)eConsequently BE o%j(E’, @ \. ‘ 


We take the mappings f,:B—>E defined by 


Theis seer Xp, ---)= fy (Xp). 


Rach of these mappings is continuous, and the set 


B= (x: fi(x)=fe(x) =... =f, (x)=...) 


is therefore closed with respect to B, i.e. B=BB , where 
B denotes the closure of Bin £’, The set B is compact, 
Therefore B= BBE o¥,,(E’. @’). The mappings f, all induce 
the same mapping f of set Bin £, Obviously f(B)cI,-for any 
n, so that f(B)ST. We show that f maps Bon to I’. For 
let c€I. There exists x,€B, such that f,(x,)=c for any n. 
The point+ = (x, x2, ..., X,,...)of EF’ belongs to B and f(x)=c, 


3°, All the open and all the closed subsets of (E, @) 
belong to &., 


For & contains any compact set I' (since we can put F’=B 
—[ and take the identity mapping for f). By 1.D, every 
closed and every open set can be expressed as the sum of a 
sequence of compact sets, Our assertion therefore follows 
from sec, 1°, 


4°, We pot reg if TES and TEY., It follows from sec, 
1° and 2° that & is closed with respect to the operations of 
denumerable summation and denumerable intersection. Since it 
follows obviously from TES’ thatTES’,/’is a s-algebra, In 
accordance with 3°, £’2>@. Consequently #/’/D5(C), i.e. it 
contains all the Borel sets of the space (£, @). 


38. Let # be any system of subsets of the semi-compact 
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(E, @) such that it contains all open and all compact sets, 

We shall call the function «, defined on ¥ and taking values 
from the segment [0, 1] , a capacity if the following condi- 
tions are fulfilled: 


3.4, IfT, Gl, and 7, LCF, then o(f,)<-(f,). 
3.B, Iff, tT and T,€%, thenTe and o(l,) te). 


3.C, Given any compact set I and any «>0, a GE@ can 
be found such that GOI and 


¢(G)—<(P) <e. 
3.D. If, T, BEF and fer, then 
¢(TUR)—e PUB) < 9(T)— (0. 
The following property is a consequence of 3.D: 


3.E, For any Pcr(i=1,2,..., of F: 


9 (U r) ety (U f) < = eV) — eT). 


For, 


where 
B=T,U...UT;iUR..Uu...-UB, (@=1,2,... 2). 


For each TCGEwe shall write ¢*(T) for the lower bound of 
o(G) with respect to all open sets G containing I, and 9.(T) 
for the upper bound of all 0 (P) over all the closed ce T 
contained in Tf, 


Qur purpose is to prove the following theorem regarding the 
extension of capacities. 


Theoren 1, Whatever the capacity ¢ in the semi-~compact 


(E, @) , we have for any analytic (and in particular, any 
Borel) set I's 


9, 1) = 9" (LT). 
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4. We shall need some lemmas before proving theorem 1, 


Lemma 2, For any sets T,eT,eF& (¢=1, 2,..., 2) 
n " aS n 
v: (U r) 7 °(U fi) < Xie) —9" Fo. 
1 ri 


If T,tlr, we have 
e*(L,) t eM). 


Proof, Let e>0Q. We choose open sets G; (i=1, 2,... 
nN 


and @ such that T:S6; G2UP, and 
1 
o(G)y<PTi)+s i¢=1, 2, oeey n), | 


9 (G)<e¢ (U +e 
1 


Land ~~ ~ 


We put @,=Ga,. Obviously G3G,> 


(09) <o(0) 


We have on the basis of (3), (4) and 3.2: 


e(U r)— #(U®) < °(U Gs) — °( j G,) +e < 
1 1 1 
<2 [9 (G)—9 (G)I+ e<d (o*(T)—9* Ty + (e+ Ye. 


Since « is arbitrary, (1) follows from (5). 
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(4) 


(5) 


Now let r,t 7. For any «>0 we can choose open sets G,, 


such that G,>7,, and 
9 (Gn) < oe = + 9" (Pn). 


nr 
We put G™ == [JG;. 
I 
In view of (1): 
2 % 
¢ (o%)— 9 ty =9/ G; —#(U n)< 
1 
ae 


1 


(6) 
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We put 
G =U G; == ij Gg 
1 1 
By 3.B: 
9{G™) t 9(G). (7) 
From (6) and (7): 
#(G)— lim ¢*(n)<e. (8) 


Since ¢(G)>2z*(P,), (2) follows from (8). 


Lemma 3, If TE #¢a(E. CC), we have 


9. (Tf) = 9* (L). 
Proof, Let TE e%3(F. @). Then I can be expressed as 


P=f)T, (p€o%.(E. @)). 


n=1 
We can choose the sets Tni€.#(E, @) such that ré+7r,. We 
assign any positive e« and put 


Bi =r. 


Obviously, Bf + T and by (2), o*(Bi) t o*(T). We select &, so 
that 


(1) — 9" (Br) < 5. 
We put 
By — Berk. 


We have BY ¢ BE. By (2), o*(B2) t o*(Be) . We choose & so 
that 


9° (Br) — 9° (BY) <4. 


On continuing these constructions, we get a sequence of sets 
ne aan Be? re, 
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such that 
® ky af * 
9 (B,*;*) —~»” (B," | een 


Thus for any 73 


* * »* E € 
e )—¢ (B,") <$+ Te ae: (9) 
We put 
ik re ke 
B=f) BR *— IT; ig : 
n=l 


— Min Kn 
A — Tr, Y, oes he e 
ad 


Since rn SVrand {)l,=T, we have 
1 


and 


is a compact set. By 3.C, we can choose an open set GDB, 
such that 


¢(G)— 9(R,) <e. (10) 


We observe that 
B=ATand A. |B. 


Hence (in view of the compactness of A,), a8 from a certain 
nyB "=A &G and by (9) and (10): 
«lo 
eK <9(Bte<eG)+e<e(B)+2. 
Since B. is compact and contained in I, we have 


?(B)<9,() and 9*(L) < 0, (F)-+ 2e. 


It follows from this, in view of the fact that « is arbitr- 
ary, that o*(T)=9,(f). 


5. Proof of theorem 1, Let Ty be an arbitrary analytic 
set of the space (£, @). We can construct in some compact 


182 THEORY OF MARKOV PROCESSES 


(E’, @’)a setBe £.,(E’.C’)admitting a continuous mapping f on 
the set I, . We take the space (EF, @) =(EXE.. @X@’) (it 
is clearly a semi-compact) and write p for the projection of 
E to E , i.e. the mapping given by 


Ds 2) (XE E, x CE’). 


The set of all points (f(x’), x’) (x’€B) will be denoted by T, : 
If p(T)€ #, we pub Te and 


(Lf) =e(1). (11) 


1°, We show that the function o(f) TE¥) is a capacity in 
the space (Ff, @). Firstly, # contains all the compact and 
all the open sets of the space (E, @). For p(P) is open if 
T is open, and compact if P is compact, Moreover, the 
mapping p preserves the operation of summation and the inclu- 
sion relationship, Hence, since conditions 3.A-3.B and 3,D 
hold for the function 9, it follows that they also hold for 
the function o, We _prove 3.C by observing that, if Tis a 
compact subset in (E, @) and if GE @ is such that 


2(G)—9eIpMl<e, 
the set G=G* E’ belongs to @ , and at the same time we have 
9 (G)—9(f) <e. 


2°, We show that Ty€o#3(F @). To start with, it may 
easily be deduced from the continuity of f that 


T,=(E x B)To, (12) 
where Ty is the closure of T, in (E. @). In accordance 


with 1.D, the closed set T) in the semi-compact (E, @) can 
be written as 


where K,€-#(E, @). On the other hand, 


E=UEm B=f) UBi. 


k=1 i=l 


where sets £,, and B, are compact, We have from (12): 


=f) U (EnXBi)K,. 


ke 1 i,-m, n=l 
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The sets (E,,< Bi)K, are evidently compact, so that T,t »,(E, 
C). 


3°, It follows from 2° and lemma 3 that 


, (Lo) = 9" (Ty). (13) 
We observe that 
%(o)= sup o> sup _ gfp(i =>.) 
Teg (E, @) Teg (Zz, @) (14) 
rer, TST, 


Gee Gee 1 
nae (15) 


G2r, 


It follows from (13), (14) and (15) that o%())>7*(f,). But 
since 9, (I'o)< ¢*([,) in view of 5.4, we have 9,(I)=9"(I) . 


2, Measurability Theorems for the Instants 
of First Departure 


6. Let X=(x,, 1, of; Ps ,) be a Markov process in the measur- 
able space (£, #). For every w€2, we write Fi=F§(w) for 
the segment of the trajectory during the time [s, t], i.e. the 
set of points +,(), where uw€Is, ¢]. 


If the process X is given in a topological measurable 
space, we shall write F§ for the closure of the set F;., 


Lemma 4, Let X be a Markov process continuous from the 
right in the topological measurable space (£, @, #). Then 
for any closed set T€ @ and any U<¢s<ct? 


{Feel} ={FiST}e WN. (16) 


Let UC @C#. We assume that one of the following condi- 
tions is fulfilled: 


6.A. (E, @. #) satisfies condition 1.9.B and the closure of 
U is compact. 


6.B, (£, @. #) is metrizable, #>C and ENU is compact, 


184 THEORY OF MARKOV PROCESSES 


Then we can construct a sequence of measurable closed sets 
lr, t U such that, for anyO<s<ti 


(FicT,) t {Figu} (17) 


and 
{Fis U} CN. (18) 


Proof, Let I be a measurable closed set and let A be any 
denumerable everywhere dense subset of the segment [s, ¢] , 
containing the point ¢. Obviously, 


so that (16) holds, 


Furthermore, let U be an open measurable set. In case 
6.4 we take the function f(x) defined by condition 1.9.B, In 
case 6.B we put f(x)=p(x, YU). The sets 


n={x:f@> 7} 


are closed and measurable and [T, ¢+u. Hvidently, 


(Alcu} aU Fen. 


n=l 


We can prove (17) simply by proving that the converse 
inclusion also holds, All we have to do for this is to show 
that 


{Fic U} & { inf f(x) > 0}. 
2eFs (19) 


In case 6,4, this inclusion follows from the fact that the 
continuous function f(x) attains a minimum on the compact set 
Fi. In case 6.B, the continuous function p(y, Fi) attains a 


minimim on the compact set U, If 
FiU =8, 
this minimum is positive and we have 


inf f(x)=p(Fi, U)> 0. 
zeRré 


t 
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As already mentioned, (17) is a consequence of inclusion 
(19) just proved. Expression (18) is a fairly obvious conse- 
quence of (16) and (17). 


7. We put AGN; if, for every measure sp on the c-algebra ¥ 
there exist sets A, and 4, of Wtsuch that 4,¢ AGA, and 


P,, pe (A;) ee Ps. 7" (Ap)* ) e 


Theorem 2, Let (E, @) be a semi-compact, let @ be the o- 
algebra in E containing @, and let X be a Markov process 
continuous from the right in the topological measurable space 


(E, C. #) e 
Then for any Borel set [ and any 0<¢s<ct: 
[FESTY EN. (20) 


Given any Borel set [ , any measure p» on the c-algebra # 
and any 0<s<t, we can construct a sequence of closed sets 


reMnc...che...2£r 


and a sequence of open sets U,DU,2... DU,3... D272 with 
compact complements such that 


Ps, {Fic r,| t Psa [Ac T}, 
Pein (eu,} i Poa thi Sf). 


Proof, We fix the 0<s<i and putl€ ¥ if 
(PPS ENT)E Wf. 


In view of lemma 4, the system contains all compact and 
all open sets, Having fixed a probability measure p on the 
c-algebra #, we put for each TES: 


9 (T)=P,. {Fil +o} =P, (2\[Ac er) (22) 


We show that the function 9 is a capacity, i.e. satisfies 
conditions 3.A-3.D. It is obvious that 3,.4-3.B are fulfilled. 


*)Cf. foot-note on p.55, 


**) This expression is meaningful because the set in brackets 
belongs to-N;G WN" and, in accordance with sec, 2.3, the 
measures Ps,» can be extended to A, 
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Conditions 3.C follows from relationship (17) which we proved 
in lemma 4, It remains to consider condition 3.D. 


We notice that, iff, 7, BE # and Per, then 
3(Ty bh) = 
=P,,.(Fiun) so}, ((Fr-zodu(Fin+oy}—= (23) 
=P, [Pi o}--P,.. {AB = o}—p,, (Pir 40, RB ¢ ole 
=2(T) +2(B)—P,, {Fil 9, FIR -¢ 9}. 
Similarly, 


9PuB)=20)+9(8)—P,,, (PY +9, Pip +o}. (24) 
We get from (23) and (24): 


—¢e(TuB)+eTuR+oM—eh= 
=P, {Ail=6, Pir +o, PIB +a} >0, 


whence 4.D follows, 


Let I be any Borel set. Then I’ is also a Borel set and, 
by theorem 1, we can find for each 2 a compact set I” and an 
open set U” such that 


rere" 
and 


0<eU)—eaM<L. 


It follows from this last expression and (22) that >aravU* 
and 


The sets 


satisfy the inclusions 


re...G,¢...STe...6u,¢6...eU,, 
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whilst 
0<P,,. {Sus} —P, (Aer) <2, (25) 
We put 
Ax=UlFicts}. 4.= {Fis up). 
n=l Ral 
Obviously, 


ASIFSI}ca, (26) 


In view of lemma 4, A, and A, belong to Nt. Furthermore, 


Bea (Te eee (27) 
Pe, y(A2) = lim P, , {FES U,|} 
nwo : 


and by (25): 
Pe, y (A) = Ps, , (Ap). 


Relationship (20) therefore holds, 


We conclude from (26) and (27) that relationship (21) also 
holds, 


Remark, The reader will observe on going through the 
proof that the theorem still holds in the case when [ is 
the complement of any analytic set, 


8. Let X=(x,, %, of, P,.,) be a Markov process in the measur- 
able space (£, #), The instant of first departure after s 
from the set [*) is given by 


=| sup {t:Fic TH, if 6 () > S, (28) 


S, if Cw<s 


(this may easily be seen to be equivalent to the definition 
given in sec, 3.8), 


*)This magnitude can also be called the instant of first 


reaching the set E—I, 
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If process X¥ is given in a topological measurable space, 
the function 


sup {t:FicYr}, if %(o)>-s, 


= % if tw)<s (29) 


will be described as the instant of first departure after 5 
from inside the set [I *) 


Theorem 3, Let X be a Markov process continuous from the 
right in the topological measurable space (£. C. #). 


If [ is a closed measurable set, we have for any 0<¢s<ci: 
{85 (LT) > t} = (8 (L) > t} EN t zo. (30) 


If T€@# and if one of conditions 6,.4-6,.B is fulfilled, 
we have for any 0<¢s<t 3 


(EP) > th EWN. (31) 


Finally, if (£. @) is a semi-compact and #2@, then for 
any Borel**) set [ and any 0<sc<} 


*)Alternatively, we can call @(r) the instant of first con- 


tact with set £-r, 

We defined in sec, 3.13 the instant of first departure 
after s from inside the open set Ir as the instant of first 
departure after s from the normal system + for which 


T= G. 
res 


if G has a compact closure, the earlier and the new defini- 
tion are easily shown to lead to the same function (cf, the 
proof of lemma 3,8), 

**)And in fact for any set I’ whose complement is an 
analytic set. 
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(8s (T) >t} € Miso *)- (32) 


In this case, we can construct for any s>0 and any measure 
»on # a sequence of closed setsl,G...€1,¢... contained 
in IT and a sequence of open setsU,>...2U,2...contained 
in I with compact complements such that 


Pein {ee Pw) t SM} =P, 2, (U,) | & (=I. (33) 


Proof, Let &,=&,("), %—=§ (Tf). We have for any v>7z and 
any integer m: 


fe. > #, 2=U (Fiat oy} (34) 
Suppose that [ is closed, Then by lemma 4: 


2 


{Pa STlew, 1. 
n , 


Consequently, 
(33 > t, 24) E ffs 


for any v>?#, so that {fs >t} EN0 , Since t= fora 
closed set, relationship (30) holds, 


Now let I’ be open and one of conditions 6,A-6.B be satis- 
fied, It follows easily from relationship (19) that we 
deduced when proving lemma 4 that 


a 


St) =F eT). 
Since by lemma 4: 


FEST} E AN, 


*)We put vcNi.0 If ASY, and 4ocAW} for any v>t . 
Obviously, ,j*}., 1s a s-algebra in the space »,. The <c- 
algebra o\7/., is similarly defined. It will be noticed that 
Aca&izo when and only when there exist for every measure » in 
the c-algebra #@ sets4’, AveN}:vsuch thata’ SAS avandP,, , (A’) = 
P,,.(4”) (ef, foot-note on p.55). 
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relationship (31) holds, 


Finally, let (F. @) be a semi-compact, let #@> Gand let I 
be an arbitrary Borel set, By theorem 2: 


73 733 
ae sve or a 
We can therefore conclude from (34) that 


{E, >t} Col i40. 


We now number all the rational mumbers 1r,, rz, ..., ln +--- By 
theorem 2, for each k we can choose sequences 


te eee eT. ¢ @ee ere Gute ..au 


(the Ty being closed, and the UX open with compact comple- 
ments) such that, as n—-+0o, 


Poi ers t Po. Fr, & I, 
Poy tPr, S Un} | Ps, {F8, ST}. 


We put 
T Eat % 
2=U Ta. Un,=[] Un 
kel kml 
Since 
ViG.csE) Cu. UpScc SU, Ses. 
and 


Mer,ercu,cut, 


for n>k, we have for each & as n->c0: 


Pe. tPr, Sa} t Poy (Ft, ST}, (35) 
Pay {Fr,S U,} + Pca r}. (36) 


A 


E£M)<..-<&0n<---<&(<--- 
<b (Un) < ++» <8 (U0) 
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so that 


(SC) rEO0}=U Nbed<n<iMje 
a. . (37) 
SU Nil-, srlNiA, srl}. 


k=1n=1 
It follows from (35) and (37) that 
Pevu (So(Pn) t &(T)} = 1. 
Similarly, it can be seen on the basis of (36) that 


ee (U,) { E, (I)} = 1. 


9. The functions &,(I) and E,(P) do not increase as s de- 
creases, with the result that there exist the limits: 


Se40(I) = lim E.(P), 
tys 

E540(I) = lim &(D), 
tvs 


Theorem 3°, Let X be a process continuous from the right 
in the topological measurable space (E, @, §). 


if I is a closed measurable set, we have for any 0¢s<?? 
(5e10(L) > t} = (e400) > #} Eto. (38) 


If TE C# and if condition 6.A or 6.B is fulfilled, we 
have for any 0<¢s<t: 


(E,s0(D) >t} EN. (39) 


Finally, if (£, @) is a senmi-compact and #D2@E@ , then 
b.40) > t} CW tio, (40) 


for any Borel*) set [ and any 0<s<t. 


In this case, given any s>0 and any measure » in #, we 


*)and for any set [ which is the complement of an analytic 


set, 
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can construct a seauence of open sets U,2...3U,3...a0I 


such that their complements U,,..., U,,... are compact and 
Psa fSs20(Un) 4 Ss0(T)| = 1. (41) 
T)—0*), we can construct a sequence of closed sets I, 


c...cY such that 
Psu Gero(ln) t bs40(T)} = 1. (42) 


Proof. Let [ be a closed measurable set. If s<uc<ct, 
we have by (30): 


G0) >t} = ul) > tH EN 0 S Mio. 


We conclude on letting w|s that (38) is satisfied for 
s<t. If we now let ¢ tend to s, it will be seen that (38) 
also holds for ¢=—s. Expressions (39) and (40) may be veri- 
fied similarly. 


Relationship (42) follows from (33) on taking into account 
the fact that Eyl) ess () for any set T and P, , {&,,,(L) =, (T)} 
=1lif .(t=0. 


We still have to prove (41). We choose a sequence s, | s 
and put 


p(T) =P, a (xe, ET} (TE #). 
Using theorem 3, we construct for each 1 a sequence of open 
sets UL,DU,D...DU,D...27, having compact complements 
and such that 
Psy “2 Lis, (Un) ‘ (f) Ns 
By (32): 
A={2, (Un) 4 i, Ty} Eel. 


iw 
2 


*)The following example shows that this condition is essen- 


tial. Let xX be a determinate motion on a straight line with 
velocity 1 and letr=(0,«). Then for any closed set TSI: 


Ps, 0 <Sn+0 (D = 0, bozo (I’) = 20} = 1. 
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By theorem 2,1’: 


Ps (A) = Mg, aPs iro (A) 
en 
so that 
P,;, . (4) = Msg, aPs Ze (A) = P... by (4) 1. 


Thus we have 
&(Un) 1 i, (PT) (ac. & Py, 


for each n aS g-+0o. We put U;=Uin... nui. Clearly , 
U,2...2U;a...37 and & (U,)<% (ui) for ron. Hence 


2, (Un) 4 4,10) (a0. 2, Py (43) 
for any 1 aS k-—>so. We have, moreover, for any x» and k, 


33, (U;,) > eee (Ux) => seg 0 (Ty 
and by virtue of (43): 


‘sy TVD iim 29 (Ux) > 20 (P) (a,c, 2. Pp 


s, a): 


Since 


lim 5. (T)= f.49(D), 


7 -> a2 


we get 


° t ° bs ee 
oF 5s40(Un) = S540 (L) (a.c,2, P, »)- 


10. Theorem 4, Let (£. @) be a semi-compact, # a s-algebra 
in £—, containing @, and X a Markov process continuous 
from the right and quasi-continuous from the left*) in the 
space (E, @, #) such that 


Niro S oh; 


for all 0<s<t., Then 
Pin s. 2,0) = 0. (44) 


*)see sec. 6.17. 
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for any Borel*) set [ and any s>0, x€E. 


Proof, Let U be any gl set. By theorem 3, a sequence 


of closed sets I,G...G7,&...€U can be constructed for 
any s>0, «EE, such that 
Po (fs(,) t (U} =). (45) 


In accordance with 1.C we can introduce a metric p(x, vy) into 
the space (E, @). We put 


F,=T,U] x: a (x, 0) ><}. 


~~ 


Obviously ,T,¢...cP,¢...q@Uand T,cf. Hence 


13,10, t EU) & (2,0) t FU) 
and 


Po {Ee@,) t e(U)} = 1. (45) 
It is clear from theorem 3 that the §, (T,,) are random variables 


independent of the future and s-past, and it follows from 
(45), since X is quasi-continuous from the left, that 


FEC) P23 (A00- 82 cry P,, 2): (46) 


But :,(T,)=?,(F,)for the closed set fT, , and *(,@,y 0) <= 
since X is continuous from the right. It follows from this, 
in view of (46), that 

6(%2 rp U)=0 (2.c. 23 wy Ps, x): (47) 
Since 

(s(U)=3,5(U)} S [XE ey EU} = [2 (28, coy YU) > 9} 
it follows from (47) that 
P,.. L £4 (U) 4%, (U)! =Q. 


This proves the theorem for open sets, 


*)See footnote on P.191. 
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Now let f be any Borel set, By theorem 3, for each s>0, 
x€E, @ sequence of open sets U,DU,D...3U,2...D can 
be constructed such that 


Pye fta(Ue)d 2.09) = 1. 


By what has been proved 


Pao fis(Un} =3,(U a) for rz=I, 2, ee 
Hence 
Ps. ta 24 (T)} ==], (48) 
But 


2.(Un) > i (LT) > 35 (0). 


and (44) therefore follows from (48). 


Remark 1, let (E,@) be a semi-compact and Pv(s,x;t,T) any 
transition function in the measurable space (£,3(@)) that 
satisfies the conditions of theorem 6.8 (sec. 6.17). Then 
it follows from theorems 6.8 and 5,11 (sec. 5.21) that a 
Markov process X exists with the transition function P\(s, x; 

t, T), for which all the conditions of theorem 4 are fulfilled, 
so that (43) holds, 


Remark 2, Let X be a process satisfying the conditions 
of theorem 4, It follows from theorems 3 and 4 that, if we 
take the c-algebras 8; in the sense indicated in the foot- 
note to p.55, all Borel sets are admissible (see sec, 3.8) 
for process X (and indeed all sets which are the complements 
of analytic sets). 


The same statement holds true without the requirement of 
quasi-continuity of the process from the left if the func- 
tions :,(f) are considered instead of functions {,(I). 


SUPPLEMENTARY NOTES 


Chapter 1 a 
The reduction of the basic ideas of probability theory to 


measure theory is due to A.N, Kolmogorov (Ref. 19). 


Chapter 1 gives a brief résumé of the results of measure 
theory which are used in the present book, A more detailed 
treatment can be found in the monographs by Halmos (Ref. 23) 
and Loeve (Ref. 20). 


Article 1, The concepts of a =-system and 4-system of sets 
and of an ¢-system of functions are introduced for the first 
time. Lemma 1,5 is due to Doob (Ref. 7, supplement, theorem 
1.6). 


Article 2, The proofs of propositions 1.4.4-1.4.C can be 
found in the works of Halmos (Ref. 23) and Loeve (Ref. 20). 


Article 3. The general definitions of conditional proba- 
bility and conditional mathematical expectation were given 
by Kolmogorov (Ref. 19). We follow Doob (Ref. 7) and Loeve 
(Ref, 20) in the formulation of these problems, The only new 
feature here is that we consider functions and s-algebras 
defined in some subset of, instead of throughout, the space 
©, (Doob mentioned the possibility of carrying over the 
normal definitions to this case in Ref, 8). 


Article 4. The definitions relating to topological and 
metric spaces can be found, for instance, in works by Haus- 
dorff (Ref. 25), Loeve (Ref, 20) and Halmos (Ref. 23). The 
concept of topological measurable space appears to be investi- 
gated for the first time in the general form, 


Article 5. Theorem 1.1 is often called the theorem on the 
extension of measure, It is generally proved in a somewhat 
different form (see, e.g. Loeve, Ref. 20, 4.1 or Halmos, Ref. 
23, sec, 13), for the case when the system @ is an algebra, 
i.e, contains the complement of each set as well as the set 
itself and the intersection of every pair of contained sets. 
This case is readily obtained from the more general case 
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considered in theorem 1,1 (see Halmos, Ref. 23, sec. 8, pro- 
blem 5). 


Theorem 1,2 is due to Daniell (Refs. 3, 4) and Kolmogorov 
(Ref, 19, chapter III, sec. 4), (See also Halmos, Ref. 23, 
chapter 9, sec. 49,) Kolmogorov stated this theorem only 
for the case when(£, @)is a straight line with its usual 
topology and #@=<-(@)3; but his proof stands unchanged in the 
more general case considered here. 


Chapter 2 
Article 1, The principle of the independence of the 


"future" from the "past" for a known present was first stated 
(as applied to sequences of random variables) by Markov (Ref. 
2). This and subsequent work by Markov reveal the actual 
history of the theory of Markov random processes, Processes 
with discrete time (Markov chains) were considered almost 
exclusively in the first period of this history. 


The foundations of the general theory of Markov processes 
were laid by Kolmogorov (Ref. 18). The transition probabili- 
ties P(s, x;t,T) formed the main subject of study in Ref, 18 
and in subsequent works. The systematic investigation of 
Markov processes in the sense of random functions began some- 
what later (Levy, Doeblin, and others). The creation of a 
satisfactory, general logical concept of the Markov process 
as a random function subject to definite requirements is 
primarily due to Doob. His work on the subject is brought 
together in the monograph of Ref. 8. 


Doob started out from a general concept of probability pro- 
cess, This is defined as a function x,(w) with values in the 
measurable space (£, #), where the argument ¢runs over some num- 
erical interval / and the argument » over some probability 
space (2, of, Pp). (It is assumed that {:x,(o)CLT}E€o% for 
any ¢t¢€/,T€@) . The condition for a process to be Markov 
is stated as: for any TE FY, t<ucil, 


P (x, C€Tol,} =P ix, E€T]x,}  (a.c.&, P) (1) 


(WN, denotes the c-algebra in @ generated by sets {wi x,(w) ET} 
(TE s€ls<h). 


Development of the theory of Markov processes in recent 
years has shown that: 


a) it is useful to consider events not appearing in the 
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s-algebra o\,, and to take as basis the strengthened variant 
of relationship (1): 


P {x €TJo#,} =P (x ET]y} (ac. &, P), 


where of, are s-algebras satisfying the inclusions #Aac#,d 
WN, A, Dot,for t>s€/3 


b) it is advantageous to allow that the process may cut 
off at a random instant {(w). 


If we take this into account, we arrive at the general con- 
cept of Markov random function given in sec, 2,1. (This con- 
cept was first considered by Doob in Ref. 8 (and see Ref. 27). 


In the theory of Markov processes it is usually a question 
of dealing, not with a single random function, but with a 
family of such functions, corresponding to all the possible 
initial instants of time and all the possible initial states. 
It is therefore natural to assume that the theory is primarily 
concerned with what we have called Markov families of random 
functions. Various special classes of Markov family have 
been discussed in a number of works but this is the first 
time that the general case has been mentioned*), We apply 
the term "Markov process" to a somewhat simpler formation 
than the Markov family. There is no loss of generality in 
taking the process as our main subject of discussion (see 
the end of sec. 2,1) and we gain at the same time in compact- 
ness and simplicity as regards the propositions and ex pres- 
sions, 


Articles 2-3. Almost all the results of these articles are 
new, The basic idea of the proof of theorem 2.8 can be traced 
to Doob. In the investigation of the trajectory properties 
theorem 2.8 replaces Doob's separability theorem (Ref, fs 
chapter 2, theorem 2,4). 


Chapter 3 


The basic ideas and results of chapter 3 are due to the 
author, The proofs of these results are published for the 
first time, Integral equation (3.76) was first deduced by 


*)The Markov random function corresponding to the transition 


function P(s,«x;7,)is quite often taken as the starting-point. 
This concept occupies an intermediate position between the 
concepts of Markov random function and Markov family. 
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another method and with different assumptions in the work by 
Blanc-Lapierre and Fortet (Ref. 1, chapter 6, sec, 14), We 
reproduce the method of deducing the equation published in 
Ref. 9. 


Chapter 4 

Theorem 4,1 can be regarded as familiar (see e.g. Ref. 7 or 
20). The basic idea of theorem 4.2 was employed, for instance, 
in Ref. 8, 


Chapter 

Up to 1955-1956 the strictly Markov property had undergone 
virtually no logical analysis, Many authors have used this 
property without any basis, apparently on the assumption that 
it is a general property of all Markov processes, 


Doob (Ref. 6) was the first to give a clear formlation and 
proof of the strictly Markov property of certain processes. 
He showed in the work in question that all stationary Markov 
processes continuous from the right with a denumerable set of 
states are strictly Markov*), Doob's result was later exten- 
ded to wider classes of stationary Markov processes with 
denumerable sets of states (Yushkevich, Ref. 27, and Ref. 28; 
K.L. Chung, Ref, 26). 


The study of strictly Markov processes as an independent 
class was begun in 1955-1956 by Dynkin (Refs, 11, 12**), 13), 
Dynkin and Yushkevich (Ref, 16) and Ray (Ref, 22), Dynkin 
showed that, by starting from the strictly Markov property 
and attaching definite requirements to the nature of the 
trajectories, infinitesimal operators can be evaluated for 
the processes and a complete classification thus obtained of 
certain important classes of process, The work of Dynkin and 
Yushkevich (Ref. 16) gives the first general definition of 
strictly Markov processes (for the stationary case), shows 
that the class of strictly Markov processes is considerably 
narrower than the class of all Markov processes, and deduces 
the sufficient conditions for a Markov process to be strictly 
Markov. Ray's work (Ref. 22) is devoted specially to con- 
tinuous one-dimensional stationary Markov processes, The 


*)The strictly Markov property is understood in a slightly 


different sense to ours in Ref, 6, and the class of processes 
discussed there is in fact a little wider than the class of 
processes continuous from the right. 

**\Detailed proofs of the results stated in Ref, 12 are 
published in Ref. 15. 
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condition for such processes to be strictly Markov is stated 
(more weakly than in Ref, 16)*) and the necessary and suffi- 
cient conditions for the process to be strictly Markov are 
deduced in terms of the transition function**). 


Hunt's work (Ref, 24) is independent of those mentioned 
above. The strictly Markov property is stated here in a form 
suitable for stationary Markov processes with independent 
increments, and the Wiener process is shown to possess this 
property. 


The concept of strictly Markov process and the criteria for 
such a process was extended in 1957 to stationary cut-off 
processes (Blumenthal, Ref. 2) and non-stationary processes 
(Dynkin, Kef, 14; Yushkevich, Ref. 28). 


The general theory which is developed in chapter 5 includes 
the basic results of Refs. 16, 14, 28, and 2 in a modified 
form, In particular, the connexion between theorems 5,9-5,10 
and the fundamental theorem of Ref. 16 is readily seen (theo- 
rem 5 of Ref. 14 and theorem 1 of Ref, 28 occupy an inter- 
mediate position between these results). The "law of zero 
or unity" (sec. 5.21) was discovered by Blumenthal (Ref. 2), 
who also first discussed the example of sec. 5.22, 


Chapter 6 
The main sources of the theorems of this chapter are the 


results of Doeblin (Ref. 5), Dynkin (Ref. 9), Kinney (Ref. 
17) and Blumenthal (Ref, 2), Im order to adapt these results 
to the definition of Markov process given in article 1 of 
chapter 2, the proofs had to be modified considerably, 


Theorem 6,2 can be looked on as a generalization of theorem 
2.1 of Ref. 2 (Blumenthal only considered the case when the 
process is given in a locally compact separable Hausdorff 
space and # ={E,} is a sequence of compact sets of this 
space). Ref, 17 contains somewhat different conditions for 
boundedness, 


*)Instead of all random variables independent of the future, 


only the instants {(a4,6) of first departure of the trajectory 
from the interval (a, 6’ appear in Ray's statement. 

**)The problem of discovering the necessary and sufficient 
conditions remains open to date if the description "strictly 
Markov" is taken in the more rigid sense of Ref. 16 and the 
present book. 
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Theorem 6,3 is a "localization" of theorem V of Kinney's 
work (Ref, 17) (Kinney takes the case [=~ £)*), 


Theorem 6.4 is due to Doeblin (Ref. 5). 


Theorem 6.5 is due to Seregin. It is a "localization" of 
the theorem proved by Dynkin (Ref. 9) and Kinney (Ref. 17). 


Finally, theorem 6.7 is a generalization of a theorem of 
Blumenthal (Ref, 2), 


The example of a Markov, but not strictly Markov, process 
described in sec, 6.18.5 was first constructed by Yushkevich 
and published in Ref. 16, 


Addendum 
Article 1, The results described in this article are due 
to Choquet (Ref. 29). 


Article 2, The measurability properties of the instants of 
first reaching the set T (or, what amounts to the same thing, 
the instants of first departure from E\[)) were studied in 
Hunt's work (Ref. 30) for the case of non-cut-off stationary 
strictly Markov processes that have no discontinuities of the 
second kind, are continuous from the right and quasi-contin- 
uous from the left, and satisfy the condition W,.,Go%;.. 

The general theorems for wider classes of Markov process 
proved in article 2 are published for the first time, 


*)A result analogous to Kinney's theorem was obtained also 


in Ref. 9, but with a tighter condition : 
a® (8) =0(Y 5). 


17. 
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